Google “cloud native,” and you’ll get more than 800 million hits. Clearly, cloud native is an important term – and people are fuzzy on what it actually means. A while ago, in a conversation with someone else “in the business”, I said, “But, it’s not cloud native!” She sagely nodded in agreement. Clearly, it means something, as we could agree on some complex constructs using that shorthand, “cloud native.” Let’s dig in more.
To define what cloud native is, we have to start with a simpler question:
What is cloud?
It’s All About Disaggregation
Nearly 50 years ago – before Microsoft had formed – Harvard students Bill Gates and Paul Allen wrote a BASIC interpreter for one of the world’s first microcomputers, the MITS Altair 8080. They put the entire program on paper tape (an early form of storage) and flew to New Mexico to show MITS what they had written.
On the descent into Albuquerque, Allen realized he had no way to read the paper tape (necessary so that the BASIC interpreter could load it onto the Altair 8080). Allen quickly wrote a program to read the paper tape. It worked, and the rest is history.
If you want to know what “aggregated” means – that’s a great example. Everything necessary to run Gates and Allen’s BASIC interpreter had to be written by Gates and Allen – including the routine to read the code from paper tape into system memory!
Fast forward to cloud computing, where the goal is to disaggregate completely. Disaggregate what? Well – everything. Compute is disaggregated from everything else, so you can purchase the amount of compute you need independently from everything else. Ditto for storage and network.
But that’s just infrastructure. Cloud also disaggregates software. Whereas Gates and Allen had to write every single line of code for their application, cloud apps today leverage “services” to handle specific tasks. Instead of years of coding massive, monolithic applications, today’s cloud apps are built with hundreds of lines of code that link dozens (or hundreds) of cloud services.
Cloud is the logical extension of the Unix philosophy of composability best articulated by Doug McElroy, the inventor of the Unix pipe, “Write programs that do one thing and do it well. Write programs to work together.”
That’s the essence of the cloud. But why disaggregate? At the simplest level, disaggregation delivers elasticity and agility.
Elasticity
The resources any workload requires vary over time. For example, James Cameron’s Avatar pushed the limits for special effects. Rendering a single frame required the equivalent of 3,000 vCPUs in the cloud for an hour. Since Avatar was shot in 48 frames per second and ran 192 minutes, there were over half a million frames to render. That’s more than 1.6 billion hours of virtual CPU cycles.
The film was in production for 12 years total, but as with all films, much of the final rendering came near the film’s release. Cloud provided the elasticity Avatar’s special effects vendor needed to get the job done. Executive VFX producer David Conley noted they couldn’t have done this without AWS.
Elasticity applies to compute, storage, networking, and just about any resource required. Cloud makes it easy and fast to scale usage up and back down again as needed.
Agility
The other benefit of cloud is agility in every sense of the word. Cloud enables:
- Development agility because of the aforementioned services architecture. Developers pull in a wide array of services and reduce coding from years and millions of lines of code to weeks and thousands of lines.
A side note is that this new “services architecture” benefits from the network effect. The more workloads use services, the more third parties are motivated to develop new services. And the more services there are, the more developers are motivated to use third-party services. This virtuous circle has accelerated the move from monolithic coding practices to service architectures.
- Infrastructure agility, because instead of purchasing, installing, and managing infrastructure, users simply request what they need – when they need it – through simple requests. The term “infrastructure-as-code” refers to using a simple “code” to provision all the infrastructure you need in minutes instead of weeks or months.
- Economic agility, because you only play for what you need, when you need it. Avatar didn’t need to build a massive sfx data center and manage it for 12 years. Instead, they simply spun up what they needed when they needed it.
So, now that we understand what cloud is, we can discuss what it takes to be truly cloud native.
What is Cloud Native?
The easy answer is that a cloud-native application interacts with the cloud using cloud-native primitives. For example, a cloud-native app would directly call Amazon AWS’s native storage services (S3, EBS, EFS, etc.). This principle applies to all cloud services – computer, store, network, etc.
For applications “born in the cloud” (i.e., designed from day one to run in, and only in, the cloud), this is easy enough. But for applications born in an on-premises world, there is a painful refactoring exercise that needs to occur. Note that this requires the application to be fully disaggregated from all infrastructure – compute, storage, and networking.
The second important requirement of being truly cloud native is to be managed “as-code.” This means being able to spin the application up with a few lines of code, as opposed to the traditional on-premises process of installing and configuring everything manually over a period of weeks.
Like being pregnant, there is no “partially cloud native.” An app either is or isn’t cloud native. If an app doesn’t fully disaggregate and embrace the “as-code” methodology, it loses the benefits of elasticity and agility promised by the cloud.
Are There Any Cloud Native Storage Solutions?
Before we talk about Cloud-Native Storage solutions, let’s review the cloud “primitives” insofar as “storage” is concerned.
It’s object storage (Azure blob / AWS S3). This is an amazingly affordable, scalable, available, durable data persistence layer with excellent performance, as measured by throughput. Its downside is that it is a new interface that is RESTful and only eventually consistent.
Then you have what I would call a poor person’s wanna-be SAN, i.e., EBS on AWS & Managed Disk on Azure. They present like a “local disk,” are resilient, and mostly behave just like a regular disk in a plain old storage server. These primitives act as the equivalent of a “protected local disk” (except it’s not attached to the local compute instance). These primitives have moderate performance characteristics and can support “random reads / writes” but are expensive.
And finally, you have instance attached NVMe SSDs, which is more like an extension of DRAM in that data on these “drives” do not persist across instance restarts, and yet they are far less expensive than DRAM or “protected local disk” while having performance characteristics between these two.
To date, the storage industry has stubbornly resisted the cloud. The legacy vendors (Dell, NetApp) have far too much hardware-specific code (deep dependency on NVRAM, tight coupling of data resiliency and data services layers) to refactor their offerings to take advantage of cloud-native primitives.
And then along came VAST and Pure, who also chose a hardware-optimized storage stack.
Pure’s cloud block storage offering is both relevant in that it uses cloud-native primitives to deliver those block data services so desired by Pure’s customers AND beyond sad! Who on earth is running on SAN in the cloud? Just boggles the mind. And while we are on this topic, I am hard pressed to understand who uses VMWare in the cloud. This also boggles the mind!
Amongst next-gen storage providers, it’s actually Weka.io that impresses. They have a “cloud-native” architecture and leverage cloud object storage as their resiliency layer. Now, if only they did not have the reputation of being an HPC niche player and being a “glass Ferrari”…
So, as CTO of Qumulo, what do I have to say about Qumulo’s cloud offering?
Stay tuned.
Mark the date in your calendars; Thursday, November 9, 2023, and be prepared to have your mind blown 🙂
