Data was the first beachhead for the cloud. Microsoft introduced blob more than a decade ago. So, it is shocking that cloud data today is so limited.
The challenge is that the cloud native storage primitive is object storage (Azure Blob), but most enterprise data is file data. And trying to store file data in the cloud as file data is painful – it is hopelessly expensive and lacks true cloud elasticity.
The challenge we faced was how to leverage the fantastic properties of Azure blob storage for cloud-based file data. It was a massive challenge, and I am excited (and more than a little proud) to announce how we have successfully accomplished that goal!
Background
File data is marooned on-prem. That’s a big problem since most enterprise data is file data. This is especially unfortunate, because file data forms the basis for AI/ML, and it is crucial to move this data to the cloud, since the world’s AI/ML ecosystem is so clearly centered there.
For AI / ML workloads, Microsoft Azure is the dominant cloud, thanks to their prescience in teaming with OpenAI (GPT4) and Meta (LLama-2).
But, file data on the cloud is limited. Existing cloud file services, compared with on-premises options, are expensive, lack enterprise features, scale poorly and deliver limited performance. Worse yet, each cloud provider’s file service is bespoke with minimal interoperability to other clouds or on-prem environments.
Organizations are forced to choose between unpalatable options – pay for expensive and limiting cloud file services or forgo their on-premises IT infrastructure and undertake the long and expensive process of re-platforming their file data applications to cloud object services. Re-platforming their file data applications to cloud object services means it takes between 18 and 36 months for businesses to start realizing the value of cloud.
Introducing Azure Native Qumulo (ANQ)
Back in April 2023, we announced the general availability of our Azure Native Qumulo Scale Service, co-developed with Microsoft. This was an industry first and was tightly integrated into the Azure portal. It enabled customers to deploy a secure enterprise file system up to 10PB in just minutes.
But that was just the start. Today, I am delighted to announce, as of Tuesday, November 14, 2023, the general availability of a significant update to the Azure Native Qumulo Scalable File Service; an enterprise-proven, feature-rich scale-out file service with a cloud native architecture, enabling us to deliver unbeaten economics and performance for the most demanding applications. ANQ’s elastic and on-demand capabilities allow customers to only pay for capacity and performance when consumed.
With Azure Native Qumulo (ANQ), we bring to unstructured data teams:
- Unbeatable economics – At as low as $30 / TB / for active workflows, billed monthly via the Azure marketplace for consumed capacity, ANQ is close to 80% less expensive than the closest alternative, and is comparable to the fully burdened on-premises cost of file storage. Check out our TCO calculator and see for yourself.
- Unlimited Elastic Performance – With 1 GBps of throughput included in the base storage cost, and a mere $0.00011 / additional GBps throughput / TB stored, metered on a per-minute basis, and billed hourly via the Azure marketplace, ANQ enables workflows to consume capacity and performance independent of each other, increasing flexibility while reducing cost. Take our pricing calculator for a spin to see what your workload will cost you.
- Unparalleled Data Durability – Based on Azure ZRS blob store with 12 9’s of data durability, and for most customers ANQ now eliminates the need for a second copy of data so common in on-premises environments.
- Cloud Service Simplicity and Speed – Unlike on-premises infrastructure that can take weeks, or even months, to acquire, deploy, and operationalize, ANQ can be configured and deployed (also torn down in the case of burst use cases) directly from the Azure service portal in a matter of minutes.
- Rich Enterprise Data Services – ANQ’s rich set of data services like quotas, snapshots, multi-protocol access, enterprise security integrations, and real-time data analytics have been proven across 1000+ customers in 20+ countries, and hundreds of demanding workflows ranging from AI / ML pipelines to security and compliance conscious use cases involving healthcare and financial data.
ANQ enables many workflows
Many organizations today have workloads that would benefit from cloud scalability and elasticity for their file workloads, and are constrained due to the limitations of cloud file offerings.
- Healthcare and Medical Imaging
ANQ can be used for either primary storage for PACS or for archiving with PACS VNA. Check out what Dayton’s Children Hospital has to say about using ANQ for their PACS data - AI/ML
I previously wrote about how Qumulo enables customers with their AI/ML workloads everywhere with our Scale AnywhereTM platform. AI / ML workloads are fueled by data, mostly unstructured data, most of which lives as files on-premises while Azure is the epicenter of the AI / ML revolution. ANQ bridges this divide! - Energy (Oil and Gas, Renewables)
The energy sector, whether traditional or renewable, involves enormous datasets for exploration, production, and monitoring. These use cases demand high throughput to meet the data speed requirements of seismic exploration, wind turbines, and reservoir simulations. - Media and Entertainment
Video editing, post-production workflows, animation, rendering workflows and teams are increasingly geographically distributed and hybrid. It’s ANQ that can make these on-prem / cloud hybrid scenarios seamless. And with Qumulo Global Namespace, we enable the distributed workforce, especially in the Media and Enterainment segment! - Genomic Sequencing
Genomic Sequencing requires HPC storage initially to capture FASTQ files coming off sequencers inside of labs and hospitals. A single run can generate as much as a few terabytes of data. Sequencing requires lots of computation to align raw data into consumable file formats downstream. Another example of a bursty use case that fits perfectly with ANQ’s elasticity. - Disaster Recovery and Business Continuity
This is an almost obvious use case for our customers. Why would you possibly not leverage Azure for your BC / DR workloads, especially with ANQ’s compelling economics
Azure Native Qumulo (ANQ) Cold
I’d also like to announce that Azure Native Qumulo Cold will be generally available in February 2024.
Based on LRS cold blob and priced at less than $10 / TB / month, ANQ Cold is the perfect synthesis of Qumulo’s enterprise file system, recognized as a leader in Gartner’s Distributed File and Object Storage Magic Quadrant six years in a row, and Azure’s LRS cold blob storage for unparalleled economics for long-term retention of active but known cold data.
While Azure cold blob storage is attractively priced for data at rest ($3.60 / TB / month), the cost of accessing data can be upwards of 10 times as much.
ANQ Cold makes this less of an issue!
All file system metadata is cached so that you can run namespace operations to your heart’s content without incurring any of the dreaded access charges. Furthermore, to simplify things for our customers, we will include data access of up to 2.5% of stored capacity at no additional charge. That’s a lot of data you can access from your cold tier of file storage
For example, with a petabyte in ANQ Cold, that’s up to 25TB of free reads per month, a value of about $750.00 each month. Translating this to more meaningful data, that’s about 25,000 high resolution medical images or 5000 HD full length movies or more than a million PDF documents.
If you are interested in a private preview of ANQ Cold, connect with us here.
Customers can use ANQ and ANQ Cold as independent services, or couple it with your data center and edge unstructured data stores using Qumulo Global Namespace, Qumulo’s unified data plane and Nexus, our unified control plane, to deliver a true hybrid IT operating model and serve globally distributed locations from a single common platform that Scales Anywhere.
Check out the following links
