Back in 2009, when I was still at Isilon, customers were clamoring for a way to tie various OneFS clusters together. Their goal was to manage all their data, irrespective of its location or protocol, through a single, unified platform.
I gathered a team of our top OneFS engineers for a comprehensive brainstorming session. Our conclusion is that OneFS could not fulfill the customers’ requests. OneFS was a kernel-based file system that lacked clean layering. As such, building what the customers desired was, at best, a herculean task and, at worst, impossible.
Fast forward 13 years, and I’m now CTO of Qumulo. Dack Busch, a talented (and deeply technical) GTM leader, said he was getting what amounted to the exact same requests from Qumulo’s customers. My initial reaction? I’ve already been through this. It’s not feasible.
But then we took a closer look and saw something had changed. Qumulo had a rich, modern user space-based file system with an exquisitely layered software architecture. Maybe…
Customers are Our Magnetic Field
That’s a core truth at Qumulo. In other words, our customer’s needs cause us to react just as a magnet causes iron filings to align. Jump? How high?
In a series of customer events in Los Angeles, Manhattan, and Toronto, the Qumulo engineering and product team dove deeply into this “ask.” Here is what we took away…
- Truth #1: Unstructured data is everywhere. We know that unstructured data is the majority of all data.
But what is new is where unstructured data is generated, managed, and consumed.
- Edge. We see this today with remote movie shoot locations, retail stores, health care points of care, and forward operating bases within our National Defense infrastructure. We now routinely see 100’s of terabytes of unstructured data sitting out at individual edge locations.
- Cloud. Enterprises burst workflows to leverage the public cloud for peak needs, use the cloud for archival and replication, and – importantly – access edge data into the cloud to build and perfect AI models.
- Core. And yeah, the on-premises data centers are here to stay, notwithstanding the most fervent initial wishes of the cloud providers.
- Truth #2: Applications – and Users – are Distributed. For decades, there has been an inexorable shift to a distributed world – users working from everywhere, and applications being served everywhere.
- Truth #3: Data Agility. It’s easy to see why providing the right people access to the right data at the right time is crucial to business success. But important is not a synonym for easy. In fact, since data is growing much faster than bandwidth, this is a brutally difficult problem that is only getting harder to solve.
-
Truth #4: Data is Broken. This was the big truth we heard. Customers are screaming to us that managing multiple silos of data distributed globally across edge, core and cloud locations is a major PITA.
It’s not that customers aren’t trying to solve this – they are. But customers simply cannot solve this themselves. They need storage vendors to craft an elegant solution from the inside out that solves this problem systemically, universally, and simply.
Fortunately, Qumulo has a solution!
Introducing Qumulo Global Namespace (Q-GNS) – The unified data plane of the Qumulo Scale Anywhere™ unstructured data platform.
For users, Qumulo Global Namespace (Q-GNS) couldn’t be simpler. Switch it on, and all your data – every bit of it – acts like it’s local. But what does that mean?
In simple terms, all your files can show up in a local directory. That means a local file, a file on the cloud, or a file residing at your factory halfway around the world all “look” to be sitting there on your local directory structure.
What if they are really, really far away, and I have a somewhat slow link? It still doesn’t matter. First, Q-GNS sports a network-optimized communication protocol that saves bandwidth and costs across locations, while ensuring timely local availability of remote data.
But there’s more packed into Q-GNS to give you the local file speed you need. Qumulo’s prefetch logic guesses what is the next data you’re going to read, and pre-fetches it for you. This reduces data transfer between locations, while guaranteeing almost instantaneous local access to remote data. Is this prefetch logic any good? Well, it’s been trained on over a decade of real-world data. So, yes, it’s pretty darn impressive.
What if other workers try to access the file at the same time? Isn’t that a problem? Nope. Q-GNS’s fine-grained read/write locking and intelligent cache management across disparate Qumulo instances means local users and applications have instant access to remote data while ensuring global data consistency.
And don’t forget, since your data is sitting on the Qumulo Scale Anywhere™ platform, you get access to all of Qumulo’s rich set of data services, like:
- Multiprotocol access
- Enterprise security integrations
- Snapshots
- Quotas
- Replication
What can you do with it?
The straightforward answer is to operate your business the way modern businesses need to function – with a Scale Anywhere™ platform that enables you to connect data, users, and applications from anywhere. This means organizations can use Q-GNS to do what was previously impossible with any primary enterprise storage solution:
- Cut costs and complexity while enhancing security by centralizing data on Qumulo, either in the cloud or on-premises, and offering high-performance local access through smaller Qumulo instances at all remote locations – edge, core, and in the cloud.
- Boost team agility by facilitating distributed teams and workflows. Q-GNS provides immediate, real-time local access to remote data, enabling Follow the Sun/Work remote collaboration pipelines.
- Simplify hybrid workflows by making on-prem data accessible to applications using cloud services. Say goodbye to refactoring on-prem applications to work in the cloud.
Example Use Cases
Here are some real-life examples of how our customers are utilizing Q-GNS today…
- Healthcare & Medical Imaging: Check out how Dayton’s Children Hospital is looking to leverage Q-GNS and Azure Native Qumulo to reimagine their PACS / PACS-VNA workflows
- AI/ML: Transfer data locally to cloud services, such as Azure OpenAI Services, for rapid pre-training and large-scale execution of big data models.
- Media & Entertainment Follow-the-Sun Production Pipelines: Provide instantaneous real-time views of extensive project directories to various global locations. “Pre-warm” and proactively cache data.
- Media & Entertainment Burst Render in the Cloud: Replicate from on-premise asset repository, and leverage popular render frameworks like Deadline to service render jobs.
- Remote access for Retail, Legal, Banking, Media and Entertainment, and Corporate Shares: Grant global employees access by transporting the data to every remote location.
- Edge Ingest in Manufacturing, Retail, Automotive, SLED/FED: Feed data from video surveillance, sensors, or vehicles to a local Qumulo that connects back to the core.
- Scientific Research & Genomics: Enable researchers from various institutions to work together on extensive genomic datasets without the need to transfer large volumes between sites.
- Data Management and Archival: Establish a high-performance cache for local access to a large-scale active archive that lives either on-premise or the public cloud.
When will it be available?
Qumulo Global Namespace is currently in pre-production private preview with numerous Qumulo customers. We are a bit overwhelmed with all the interest and cannot accept any additional private preview customers at this time.
It will be broadly available, production-ready, in a public preview mode in Feb 2024. This first iteration will support performant and consistent read-only local access to remote data.
We expect to roll out performant and consistent read/write local access to remote data in 2H 2024. Contact us (or DM me via LinkedIn) if you want to participate in the private preview of this version.
What will it cost?
We have not yet finalized the pricing model. However, suppose you, as a customer, deploy this into production before we finalize pricing? In that case, we’ll allow you to continue using what you’ve deployed into production for free, as long as you remain a customer in good standing, as our way of saying thank you. Consider this our way of expressing gratitude to you, our valued customers, without whom we would serve no purpose.
Customers can use Qumulo Global Namespace as an independent service or couple it with ANQ & ANQ Cold to deliver a true hybrid IT operating model and serve globally distributed locations from a single common platform that can Scale AnywhereTM.
Introducing Nexus
I’d also like to announce, as of Thursday, 7th December 2023, the broad availability of Nexus, the unified control plane for the Qumulo Scale Anywhere™ unstructured data platform.
A cloud based service, Nexus provides a single pane of “glass” to securely monitor and manage all of your Qumulo systems, no matter where it is, edge to core and the cloud. As a cloud-based SaaS offering, expect to see updates in features and functionality on a bi-weekly basis.
And yes, it is included in your Qumulo subscription.
Check out the following links to learn more
