Earlier this month, Cisco released the their Global Mobile Traffic Forecast Update. The report details how data use from mobile phones and predicts where future growth. And the numbers are staggering. The most startling information is this. Traffic grew by 63% in 2016, reaching a total of over 7 exabytes (EB) per month! And that is only where we are at now. According to the report, by 2021 we can expect the average mobile device generate 6.8 GB of data a month. Now that is a lot of data.
Download a case study of how Qumulo helps one of the world’s largest telecom companies.
The race to provide the best unlimited data plan to customers heating up this year. It is incredibly important that wireless companies maintain constant up-time from their IT infrastructures. Imagine how frustrating it is when a video or webpage won’t load on your phone. That is the feeling wireless companies work tirelessly to prevent. To do that, providers process log data daily from billions of endpoint connections. It helps measure certain events, such as the Super Bowl, or identify anomalies that could indicate an outage.
Unique Challenges of Storing and Managing Telecom Data
Ingesting, storing, and processing these log files creates terabytes (TB) of data each day, leading to an infrastructure housing multiple petabytes (PB). Made of many small files, data sets like this can pose a challenge of managing mixed workloads. And in order to help make sense of the overwhelming amount of data, many telecom companies are using tools like Splunk for analysis. Companies are still looking for the best way to store, curate and manage data sets on this scale.
Just recently, one of the top U.S.-based carriers was thinking about this very problem. There were three options to get their storage into the modern era of machine data. They could either:
- Remain with their legacy storage system
- Explore making the move to object storage
- If they were lucky, find a new vendor offering innovation
Quickly eliminated was the first option. The legacy system that was in continuous use and hadn’t been offline in three years. The fear was that once it did go offline, it would never come back up. The idea of moving to object storage was attractive because the scalability and programmability available via an API.
However, the business had come to rely on standard file protocols like NFS, and moving away from that and having to reprogram all of their applications seemed like a daunting task that could slow down daily operations. Overcoming the challenge of finding a storage system capable of meeting their needs was starting to look impossible. “When you’re ingesting TBs of data each day from more than 60 billion incoming events, and hitting processing peaks of 25,000 IOPS for analyzing data, you need storage capacity and performance that is very, very scalable,” noted one of the company’s high level executive. “You can’t afford a misfire with the production system.”
Finding Storage That Fits For Telecom Data
What telecom workloads like this really need is a storage system with the scalability and API programmability of object that does not sacrifice the reliability of standards based scale-out systems. Fortunately for this global telecommunications carrier, Qumulo is bringing innovation back to storage by doing just that.
“We loved the innovation and energy of [vendors like] Isilon’s early days, and that’s what we saw in Qumulo. Right now, Qumulo’s the closest thing to an Apple unboxing, setup, and support experience in the storage world”, said the same high level executive. Like other scale-out systems, Qumulo’s scalability of both capacity and performance is critical in telecom environments, specifically for data reads.
With multiple processes and analysis tools run against the data, the system is constantly pounded for reads. But in addition to expected reliability, the telecom company now has unprecedented storage management capabilities with Qumulo. With real-time data visibility, the team can monitor and manage usage and see capacity and performance trends. Add to that the fact that Qumulo is entirely programmable via a REST API, and what you get is a clean, intuitive web interface that helps with usability. When asked about the Qumulo experience, a senior engineer said “honestly, the management process is painless. I couldn’t ask for things to be any easier.”
Qumulo for Telecom Data
All-in-all, the carrier migrated all of its data over a 30-day period to their new Qumulo production cluster, which currently hosts almost 2 PBs of machine data and is growing at a rate of more than 2.4 TBs a day. And the solution seems to be working, as the carrier is in the process of adding another 4 PBs of storage to mirror the production cluster for disaster recovery.
Even if you aren’t managing billions of log files, machine data is impacting IT environments across industries. If you would like to talk with one of our storage experts about how to solve modern storage challenges, send us a note and we will reach out.
