Big Data Means Big Traffic

Now is the Time to Plan for Your WAN

May, 2014

 

Organizations deploying big data must be ready to handle a corresponding surge of big traffic on their networks as they collect, transport, store, analyze and retrieve data. In a survey of IT managers, Cisco Systems found that 71 percent expected their network loads to at least double over the next two years. However, only 40 percent reported they are ready for a surge in network traffic.1 To stay ahead of the curve, one in four companies are already increasing network bandwidth to prepare for the flood of big data.2

“These workloads are bringing the WAN to the breaking point,” concludes research firm IDC. ”Network throughput is paramount to the successful implementation of big data in transactional environments.”3

Big data storage is scaling from gigabytes (GB) to terabytes (1 TB = 1,000 GB) and petabytes (1 PB = 1,000,000 GB). According to a 2014 IDC survey, the average organization now stores 164 TB of data, with that number expected to skyrocket to 289 TB within 18 months. Nearly a third of the biggest enterprises expect to be working with a 1 PB dataset by the end of 2014.4

Not only are companies managing more data, they are storing it in more places. A survey by cloud service provider Navisite, a Time Warner Cable company, finds that 59 percent of organizations store their data in three or more geographic locations.5

Hadoop, an open-source software solution for large-scale data storage and processing using commodity server hardware, has become a high-tech superstar. While companies like Facebook have created a buzz with massive Hadoop implementations, organizations are deploying big data with a variety of storage and processing architectures.

Researchers estimate that 0.5 Gbps of network bandwidth is needed per PB of big data under management by Hadoop. One network engineering consultant observed that, for many customers, Hadoop traffic alone accounted for 20 percent or more of WAN bandwidth utilization between data centers.6

The explosion in distributed big data storage correlates with surging network bandwidth. Now is the time to plan for the big impact of big data on your WAN.




Sources:

1 Cisco Systems, ”Connected World Technology Report,” 2012.

2 IDG, Enterprise Big Data Study, 2014.

3 IDC, “The Intelligent WAN: IT Transformation, Content, and Big Data Pose Big Traffic Challenges,” February 2012.

4 IDG, Enterprise Big Data Study, 2014.

5 Navisite Storage Survey, 2013.

6 Ashish Shah, “Big Traffic Networking: How Big Data Impacts WAN,” August 16, 2012.