TwinStrata

What are some good reasons to adopt cloud storage? Cost, durability and flexibility.

So let me talk about performance, instead.

Look at this graph:

As part of our daily testing, we do routine performance measurements across a broad swath of cloud storage providers. It gives us a check to ensure that the various CloudArray subsystems are performing as they should, and gives us the data to make optimization decisions. In this particular test, we measure transfer rates at various buffer sizes. We “fill the pipe” by queueing up multiple streams of data simultaneously, initiating one transfer as soon as the previous one finishes, so that latency doesn’t skew the data.

This particular provider reaches its peak bandwidth at 256K transfer size, and is actually transferring at 50% of peak at 32K.

But look at this graph:

This data is from a Microsoft study published in FAST 2007. It describes the distribution of file sizes in a file system — interestingly, even though the mean file size does creep up over the years of the study, the distribution doesn’t change much. What we can draw from it is the fact that in a typical file system, roughly 80% of the files are less than 32k.

We can see that a naive system which just maps files on the file system directly to objects in the cloud is going to spend 80% of its file transfers at the bottom half of the bandwidth curve, achieving less than 50% of the peak available bandwidth. That’s ignoring latency, too: our hypothetical naive system is going to have to be streaming files out at a perfectly synchronized pace in order to achieve the theoretical maximum.

Our Provider A actually does pretty well with small transfers, rising to peak bandwidth relatively rapidly. What about a different provider?

What abysmal performance, right? Provider B’s bandwidth only rises to almost 20% of Provider A’s by 512K. Forget about issues with small writes: what reason would anybody have for picking Provider B? Could any cost or durability benefits be enough to suffer performance penalties this big?

But let’s zoom out and take a bigger picture view:

Oops. This graph tells an entirely different story. There’s a real performance benefit to using Provider B, assuming that you are transferring large chunks of data.

Every cloud storage provider has different characteristics, even if the APIs are similar. The role of a cloud storage gateway is to smooth over the differences and provide a predictable solution for storing data in the cloud. That’s what CloudArray does: aggregates lots of small writes into large-block transfers, absorbs transient failures and network faults, and generally works to manage and optimize cloud storage utilization.

If a gateway vendor tells you that they won’t work with a particular cloud storage provider because its performance doesn’t meet their SLAs, then the simple fact is that their gateway isn’t doing its job. The more complicated fact is that naive file-to-object mappers will always be fundamentally flawed when dealing with real-world business data, because real-world business data is housed within file systems, and file systems are designed to talk to storage subsystems, and storage subsystems are designed to talk to disk controllers.

Anybody who’s worked in the enterprise storage array business can tell you about the small write problem in RAID: here it is again, written in the clouds. You’d pay a penalty for writing small pieces of data to a RAID volume, except for the years of work that have been spent developing storage systems that smooth out performance without sacrificing reliability. Odds are that many users have never heard of the small write problem, much less tuned their software to it or tried to plan out their file systems around optimizing their storage arrays.

But that’s exactly what they’ll need to be doing with their cloud storage, unless they use CloudArray. What’s our secret sauce? Actually, it’s no secret: we’re a block storage device, and we can tune and perfect our transfers to match your provider. That means we minimize time-to-durability and maximize the effectiveness of our cache. And that’s why we can make pictures like this:

We don’t make recommendations or publish our performance test results because ultimately, the choice of cloud storage provider should be a business decision, based on business factors like cost, durability, location, and a host of others. CloudArray’s architecture and capabilities make it possible for our customers to make those decisions for themselves, while being assured of getting the best performance.

– John Bates, CTO

Footnote/Mathematical aside: a careful reader will note that I discussed 80% of transfers, not transferred data. In fact, depending upon the total number of files and the distribution of sizes of the upper 20%, small files may be less than 1% of the total used capacity. And therefore, given the totally unrealistic model which disregards latency, a system with a low ratio of file count to total capacity and with a provider performance profile like A’s would pay only a minor small write penalty.

But that just serves to strengthen my point: why should any of this matter you, the user? A system with a higher ratio, or with a high-bandwidth skewed provider, can wind up spending 60% of its time transferring 10% of its data. Why should it be up to you to calculate your file system distributions and match them to the right cloud storage?

Bookmark to:
Del.icio.us | digg | Twitter | MySpace |
FaceBook | reddit | Stumble Upon |

There are many excellent reasons to use cloud storage, but fast and efficient transfer of large amounts of data isn’t usually listed as a benefit. That’s one of the reasons why people use cloud storage gateways: to speed up cloud storage access. Recently, I realized we’ve never published any details on the performance gains that one should expect when using the CloudArray storage gateway, so I decided to create a simple illustrative test. In this article, I describe the results and explain some cloud storage implementation details that contribute to performance differences.

I came up with a quick test: copy one gigabyte of fully random data to the cloud, broken up into 32768 32k files. The questions are, how long would it take for a user to copy that much data to a CloudArray volume, and how soon before all of that data is safely stored in the cloud?

To set up the test, I used an old, slow laptop to run CloudArray. (That’s what I had handy in my office…) I used just the default basic configuration of a 25G cache to configure a single 100G encrypted volume, attached to Amazon S3, and mapped to a linux client. I formatted the volume with an ext3 file system and used the ‘cp’ command to copy the files. As a basis for comparison, I used an open-source file transfer utility, CyberDuck, to transfer exactly the same files to exactly the same the S3 account.

I’ve broken the CloudArray results down into two events: the data was “usable” when the ‘cp’ operation completes, and “complete” once all of the data had been copied to the cloud. At the usable point, any host attached to the CloudArray could access any of the files, as they were all stored in the local cache. In the background, the CloudArray busily pushed the data to the cloud, and when it was done, I considered the test complete. Of course, for the transfer utility, there’s no such split. The data was either copied to the cloud, or not.

The results of running the test were, well, dramatic:

For the record, that’s 6:22 (min:sec) for the CloudArray to reach the usable state and another 3:19 to reach the complete state, while it took the file transfer 110:05 to transfer exactly the same data.

What’s the reason for the huge difference? Well, I confess that I did set up this test to highlight one of the advantages of CloudArray; because we’re block-based, we’re not sensitive to the kinds of problems that plague file-based approaches.

The caching accounts for the rapid time to usability, but the more significant part of the equation is the aggregation that CloudArray performs due to the block-level IO. We send out all data in large, cloud-optimized chunks. Cloud storage provider systems store a single 1M object faster than they store a 32k object, so sending them 1024 1M objects is easier for them to handle. That same arithmetic applies to the number of requests, so that we sent 1024 PUT requests as opposed to the 32768 PUTs that CyberDuck (and, indeed, any file-based cloud utility) must send to handle this particular workload.

In other words, regardless of the size or number of files that you store on a CloudArray volume, traffic to the cloud is optimized to give the best performance.

To test read performance, I just copied the same files back from the cloud. Using CloudArray, that once again gave two separate cases: either all of the files were already in local cache, which meant that it was roughly the same as a local file copy, or they weren’t, in which case the data had to be read back from the cloud. Again, the data was automatically read from the cloud in large chunks, giving the best performance. The copy was performed with the ‘cp’ command on the linux host, and compared to CyberDuck transferring the files back.

The results are actually pretty symmetrical with respect to the original write performance:

Copying all the files from the local CloudArray cache took 6:05, while invalidating the cache and doing the same copy again took 11:42. On the other hand, the utility transfer took 112:24.

None of the numbers that I’ve given are all that useful for comparison with other environments. A large number of factors affect actual performance, e.g. WAN speed, LAN speed, and local disk speed, and I made no attempt to optimize any of them. A faster local disk or SSD, for example, would substantially reduce the time to usability. The importance of these results is the relative performance of CloudArray when compared to the raw transfer, and those results are impressive: 17.3x faster to usability, 11.4x faster to durability, and 9.6x faster to reload (18.4x if the data is in cache).

That’s minutes versus hours.

It’s a quick cup of coffee versus a three-martini lunch.

It’s renewing online versus waiting at the DMV.

It’s time saved.

Bookmark to:
Del.icio.us | digg | Twitter | MySpace |
FaceBook | reddit | Stumble Upon |