Sunday, March 16, 2008

Bandwidth, one hurdle in adopting Cloud Storage

This weekend, I read NY Times article Video Road Hogs Stir Fear of Internet Traffic Jam.
Last year, by one estimate, the video site YouTube, owned by Google, consumed as much bandwidth as the entire Internet did in 2000. …

In a widely cited report published last November, a research firm projected that user demand for the Internet could outpace network capacity by 2011. …

Moving images, far more than words or sounds, are hefty rivers of digital bits as they traverse the Internet’s pipes and gateways, requiring, in industry parlance, more bandwidth.
While reading the article, it occurred to me that isn't bandwidth going to be the main hurdle in adoption of storage in the cloud. When clients are not happy with 10/100/1000Mbps connection with application/server/data center, how can they be happy with DSL/Cable/T1/T3 connection to the cloud? I am sure everyone has felt the pain of trying to transfer large datasets over the Internet.

If you review the introduction and growth of various Amazon Web Services (AWS), a comparatively established cloud player, you will notice very limited use cases of Simple Storage Service (S3) on its own with clients outside the cloud. Most S3 usage is fronted by another AWS in the cloud such as Elastic Compute Cloud (EC2). Such combinations overcome the challenge of transferring large amount of data between storage cloud and an application/server outside the cloud over Internet. For cloud storage to be successful, it need to be in the same cloud with application/server or connected to application/server cloud with high speed link.

Any technology that can reduce the data transfer between the cloud services and clients outside the cloud will be the big beneficiary in this trend. Caching, Compression, and Data De-duplication will most likely benefit in the near term. And, the future seems to be very much like the past aka mainframe - Desktop Virtualization, Streaming, and On-the-Fly Visualization.

So, how will new cloud players like Nirvanix, EMC Mozy and Rackspace differentiate?


  1. Good points Anil. I think differentiation will come in the form of response time and security services. But both of these are hard to prove and build a reputation around. Corporate storage customers are very conservative, this business is much more difficult than it appears.

  2. Hi Anil, I think the limits are somewhat self-imposed structural issues. Seems like the broadband industry is squeezing the pipe at both ends.

    Doesn't mean there won't be amazing growth and a challenge to fulfill...but there is more capacity available than our current system belies.

  3. How about tiering? Cloud storage can be another tier, perhaps one with lower service level requirements. This will fit into vendors' existing ILM strategy. I also think advancements in network acceleration technologies from companies like Riverbed will help the adoption in the long run.

  4. Marc,

    Agree with you on difficulties with cloud business specially migrating activities to cloud for the sake of "fad."

    Considering corporate customers are now willing to store their precious asset - sales leads and customer info on Salesforce, overcoming security and response time concerns are not as great of a challenge, unlike StorageNetwork days.

    IMO, as long as great value, not easily attainable, is added by moving to cloud, corporate customers will look at the cloud services offering point solutions targeting specific pain points such as backups for mobile employees.


  5. Pete,

    Even if broadband industry opens up the pipe, I am not sure that WAN bandwidth will meet or exceed the available LAN bandwidth anytime.

    No doubt that cloud offers great promise but what opportunities does it offer for storage industry to profit from? :-)


  6. Tyler,

    Interesting, matching cloud storage with low service level requirement. IMO, moving any existing tier to cloud without adding significant and unique value will have same question "Is it really worth it?"

    WAN acceleration technologies will help in overcoming bandwidth hurdles between cloud and rest of the universe and actually may aid in cloud adoption for traditional usage.

    Thanks for your comment. Please let me know if you come across anything interesting in WAN acceleration area already helping cloud.


  7. This is how the anti-net neutrality folk want the American populace to think. They want us to believe there is going to be a point in time when the Internet is no longer able to supply the bandwidth we demand.

    The simple truth is that we live in a country where the telcos have enjoyed a government supported monopoly from the start. These corporations have NO idea how to compete in a competitive environment; they only know how to operate as monopolies.

    Asia is outpacing us in terms of Internet bandwidth to the home (how does 100mbps sound to you!). Search for "South Korea 100mbps" or "Japan 100mbps", and then try "US 100mbps" or "USA 100mbps". You'll notice immediately that these other countries have bandwidth to burn, and the US searches will find information on NIC cards and the like.

    This scarcity of bandwidth caused by the providers is only going to see technology innovation leave the US. People in other countries can do what we only DREAM of doing. Given those abilities, they are going to innovate in ways we do not even begin to DREAM.

    These other companies in these other countries are delivering high bandwidth. Why aren't we?