A Life Sciences Institute maps the internal connections of the brain. Another research group uses cyber archeology sensors to locate excavation sites. A visual effects studio renders graphics for movies like Star Trek Into Darkness and Deadpool. While the work of these organizations is very different, each are facing a similar machine data challenge. This challenge comes from a shift in today’s work environments: the largest producers of data aren’t humans, but machines. And since machines don’t sleep, and won’t be dreaming of electric sheep any time soon, they can create a lot of data.
At Qumulo, we see it as our mission to help enterprises overcome machine data storage and management challenges. That’s why our mission is to be the company the world trusts to store, manage, and curate its data forever. We went as far as to paint it on our walls!
But there is a difference between what we say and what our users say. So to find out how well Qumulo Core works in the real world of big data, the Taneja Group spoke with customers from life sciences, media and entertainment, telecommunications, higher education, and automotive industries. What they got wasn’t simply product testimonials, but rather direct insights from storage and systems administrators about this new age of data. Essentially, the six interviews are a toolkit for solving machine data challenges.
Just as we learned from talking with industry experts, the Taneja Group learned from our customers that there is no single feature or add-on to a storage system that will make it an ideal solution for machine data. What they did find, however, is there are a number of attributes that, when working together, can help manage the onslaught of machine data:
The nature of today’s IT environment is complex as it is. Any tool that can alleviate from that is a welcome addition, and storage is no exception. More than anything else, our users say they are after a storage system that is intuitive and easy to use. But don’t take it from us, or the Taneja Group for that matter.
One user from an automotive company described how ease-of-use in storage leads to ease-of-management for their machine data like this: “We create a share every now and then and the occasional update, but that’s all. Sometimes, we have to track down a problem using Qumulo’s real-time analytics. It’s not always a storage issue by any means, and now it is much easier to prove.”
Put even more directly, a research center at a major university said they “needed something that would simply plug in and work well.” On an team that does not have a dedicated storage administrator, having a simple storage interface is a huge benefit. And with the right solution in place, they now spend less than four hours a month supporting the cluster.
For many, looking for storage always involved finding a compromise between small and large file performance. Some systems are great at one, but almost never at both. But with different machines running different applications, finding a way to make that compromise work is itself becoming a challenge.
At one media and entertainment company, their most data-intensive operations involve animation design and rendering. With 24 render nodes working around the clock, the team recognizes that storage is a crucial part of their IT environment, saying “the [render] nodes are hitting the Qumulo twenty-four by seven. We haven’t had any problems.”
At a life sciences research institute, their performance needs came from a large HPC infrastructure – 288-server cluster with 10GbE connections – that is used for things like large volumes of raw microscopy data. “An HPC cluster can pump a lot of data into a system really fast” they said of their workloads, “Qumulo has had no problem”.
In total, the Taneja Group talked with six companies about their machine data challenges. To get the full details from their report, download a copy here. But if you are more of an audio/visual learner, we also held a webinar with the Taneja Group and one of our customers interviewed for the report, which you can view on-demand here. And as always, you can set up some time to talk with directly with storage experts about your own machine data challenges.