In this 3-part series, we cover the ten most common data storage pains in large-scale enterprise environments and how to resolve them. The first article covered the pains of lack of capacity, performance and scaling, and the second article dealt with the pains of legacy systems, availability, and budget.
Dealing with “data blindness,” data loss, data locality, and migration to the cloud storage pains
In this final entry in our series on how to overcome enterprise storage pains, we’ll tackle data blindness, data loss, data locality, and data migration.
7. Data blindness pain
A lot of storage systems aren’t great at data awareness. They’re kind of dumb, or mute for a better word; most storage systems don’t tell you about the specifics of the data or what clients are doing to it right now. It’s possible to get answers to those questions by other means – but all of those means introduce complexity.
- Your storage system should be able to answer specific questions about your data such as: What is consuming all of this throughput? Where the heck did my capacity go on Sunday? What is eating up my capacity right now? What do I need to back up? What can I safely archive? When am I going to need more storage?
- Most higher-end storage providers offer visibility tools. In any case, storage is the thing best positioned in your data center to tell you things about itself and the things that access it.
- To get a handle on your data, research data visualization tools – do they answer the questions you’ve had recently when evaluating your incumbent solution? Can you monitor and control data access and use in real time?
- If you value integration with your management system, you should demand API access. If your storage vendor isn’t providing access to it, ask them to.
“Moving from no analytics on the legacy systems to rich analytics on Qumulo has been a game-changer. Accurate and real-time trending information improves decisions, which saves lives.” – David Emery, Supervising Systems Administrator, Riverside County Sheriff’s Department
8. Data loss pain
It hurts me to say it. I cringe even to say the words “data loss.” To state the obvious, data protection is very important. In this industry, the data is the actual thing we work on and modify – so lost data is lost time, lost money, lost jobs. And in the case of a successful ransomware attack, it can mean a loss of business continuity. For these reasons, you’re going to want to look at systems that help you protect your data storage really well as part of a holistic security strategy. Here are some suggestions:
- Make sure “rebuild performance” aligns with “drive population.” If it goes down, it’s moving in the wrong direction. It needs to increase as drive numbers go up. There needs to be some sort of parallel rebuild system.
- Stay on the minimum protection level possible – don’t jump in at a low level with the goal of wanting to protect everything; it’s going to cost you and increase the cost of small random writes.
- There are some object and scale-out systems that do per-file data protection. Avoid that if you can. If you have a small file count, it might not be a big deal, but as file counts grow, that strategy won’t pan out.
- Make sure your business continuity strategy includes a disaster recovery plan that adds an additional layer of defense with backup of data storage in the cloud.
“Some NAS solutions perform well on large files, others on small ones, but Qumulo was good on both. This was the first factor that really counted for us because it meant we could perform computing directly from the storage array. No more jumping back and forth between the array and the servers, with all the risks of losing data this entails.” – Vincent Julié, Chief Operating Officer, Portalliance
9. Data locality pain
Does this pain resonate with you or someone you know? It goes like this: “I have data on-premises that also needs to be in the cloud! In this case, I made a mistake moving it to the cloud and now I want to repatriate my data! I have multiple sites and I need to share only one namespace between them!
In this connected world, more and more businesses of all sizes are opening multiple locations, whether they are physical or virtual. Users at different locations and performing complementary functions need access to the latest datasets in order to collaborate efficiently.
- Make sure data is easy to move around using replication, and/or optimized data transfer.
- Make sure the file system vendor can run their software on premises and in the cloud effectively.
- You’ll want to favor flexibility in your storage solution beyond the scalability mentioned above. Ease-of-data movement via tools like replication or caching or tiering can provide high value if you have multiple locations, physical or virtual.
“Data is the currency and lifeblood of scientific research. We needed to respond to the concerns of exponential data growth, how to store and present data in an intuitive and non-intrusive way, and how to manage and protect stored data without impacting daily operations.” – Jason Krish, Director of Information Technology, Fralin Biomedical Research
10. Data migration to the cloud pain
Understanding “what” makes sense to move to the cloud can help you avoid the unpredictable cost pain, the data locality pain (mentioned above), and the shared responsibility for security pain. It sounds complex but it’s really just a matter of common sense. Which high-performance workloads make sense to run in the cloud versus on premises? Your use case should help you determine whether to run on a private cloud, public clouds or a hybrid-cloud environment.
However, for the ultimate flexibility, you need a file system that supports workloads on-premises and in the cloud. Ask if your storage vendor has a cloud strategy, can they run the same software in the cloud, is it easy to move datasets into and out of the cloud? What file access protocols do they support and does that cover the apps you need to run in the cloud? Can their filesystem scale capacity linearly with performance? And, can you run your high-performance workloads at the scale you need or is there a limit? You can explore the tricks and treats of migrating to the cloud — but the key is flexibility.
- Just say no to data minimums and caps. You can run in the cloud and store and manage as much or as little file data as you choose – whether you run on a managed service or manage your own workloads in the cloud. Choose carefully-and you won’t need to worry about data minimums or caps as your workloads change.
- Make sure you have the flexibility of a hybrid cloud strategy so you can run on a private cloud, hybrid cloud or in multiple public clouds and take advantage of what different cloud providers offer, from cloud-native services to managed cloud services to industry-specific virtual environments. Flexibility also means multi-protocol file access to a single namespace (file data lake) that is easily accessible from your data center and multi-clouds.
- Insist on controllable, predictable costs–by making sure your file system or managed service provides the ability to control how much data goes onto the cloud with a set price per TB. In this way you can avoid surprises with predictable expenses.
- Many enterprises take their first step to the cloud by creating a backup and disaster recovery second-site in the cloud. This allows them to have backups of their most sensitive data away from the data center in case ransomware or a natural disaster compromises their primary site. But to do this, your file system must make it easy to move your data to and from the cloud.
- Before you move to the cloud, find out what the storage vendor’s track record is for customer satisfaction. You should not be left on your own to figure it out. Check their NPS scores to see how their customers have rated them.
- Don’t go it alone. See if your data storage partner has an experienced partner ecosystem of integrators from which you can choose to help you create and execute on your cloud migration strategy.
All Gain, No Pain
If you’re having storage pains in your environment, contact us to set up a meeting. We don’t need a universal pain scale to help you figure out how to resolve common storage pains. Our distributed, scale-out NAS file system was purpose built for hybrid-cloud environments and to support high-performance workloads at massive scale — wherever you choose.
Learn why Qumulo was named a 4 time Leader in the Gartner® Magic Quadrant™.
Four Quadrants. Four Years. One Position.
Learn why Qumulo was named a 4 time Leader in the Gartner® Magic Quadrant™.Learn More