Helping Progenity Plan For Its Future: Service, Value and Scale with Qumulo

Progenity, Inc. is a privately-held biotechnology company that provides clinicians with complex molecular and specialized diagnostic tests for women’s health, reproductive medicine, and oncology. Progenity conducts research in areas like genetic carrier testing, CFDNA screening, and cancer testing to encourage the spread of information and ultimately, provide the industry with data that leads to better patient care.

The company was founded in 2012, currently has more than 500 employees, and is growing rapidly. In addition to its headquarters in San Diego, it has laboratory and operations facilities in Ann Arbor, Michigan and Dallas, Texas.

Their genetic sequencing operation, which produces billions of small files, is all housed on Qumulo software.

Growing data-intensive workloads requires efficient, scalable storage

As the company continues to expand, its data storage needs are also growing exponentially. Over the years, Progenity’s work in genetic sequencing has generated more than a billion files. According to David Meiser, Solutions Architect for Linux and Windows applications at Progenity, “That pace is accelerating. Within two years, we might have another billion files.”

With its rapid growth and data-intensive workflows, Progenity knew that its previous storage vendor would be unable to meet its future needs. “After a few years with our original storage system, we realized that the way the company worked wasn’t a good model for us,” said Meiser, referring to both high costs and storage efficiencies.

“One problem that was always present was that there was significant file overhead,” said Meiser, “The files we write are very small, and the block size of our old storage system was very large.” Further, Meiser explained, “We found that we couldn’t do analysis in-place because the access times were super high.”

Value today and in the future, whether on-prem or in the cloud

Progenity selected Qumulo’s hybrid cloud file storage system to replace its legacy vendor in 2016. Qumulo was able to offer an affordable solution for Progenity, without sacrificing performance or scalability. Qumulo storage also runs both on-prem and in the public cloud, another key advantage, as Progenity is planning for an eventual migration to the cloud.

“Qumulo handles small files very well, and that again goes back to cost because we’re getting better cost density. Even if we were paying the same for a gigabyte of data on Qumulo as we were for our legacy system, we’d still be getting a better value for our money,” said Meiser.

Meiser estimated that, conservatively, Progenity saves 13 percent on storage space with Qumulo compared to its previous vendor, but he noted that the actual savings is probably more between 17 and 20 percent.

Why Qumulo?

Real-Time Visibility
Qumulo’s ability to provide real-time visibility into data usage, including hotspots and inefficiencies, has proven to be an important benefit. “Now that we have it, we very much appreciate it. As we were migrating from the legacy system to Qumulo, we were able to figure out which files the research and development folks could delete so we weren’t wasting storage.”

Multi-Protocol Support
Qumulo’s support of both network file system (NFS) and server message block (SMB) protocols is another key advantage for Progenity. Meiser said, “We’ve got to be able to write data over SMB and read that data back over NFS. That’s our big story for how we analyze data.”

Progenity’s data inputs come from Illumina sequencers, which deliver results directly to the Qumulo cluster over SMB. The data is also copied to the NGS (next-generation sequencing) compute cluster, which includes local storage used for the actual computational analysis using the NFS protocol.

An Easy Path to the Cloud
Progenity is planning to move its next-generation sequencing (NGS) compute cluster to the cloud, and to use Qumulo for AWS for its public cloud storage. Qumulo’s continuous replication feature ensures that local shares will be kept up-to-date.

Currently, Progenity’s on-premises NGS cluster is underused – it’s utilized only about 50 percent of the time. “If we could move that local work to AWS, we’d end up saving money. On-prem, the compute is just consuming power and resources. If we were to move it to AWS, we would only need to pay for the resources we use,” said Meiser.

Ensuring Success with Qumulo Care

Meiser is confident that Qumulo will always be there to address his team’s questions and concerns. “We’ve been exceedingly happy with the answers we’ve gotten from Qumulo at all levels. One of the biggest selling points is having a Slack channel where I can immediately get in touch with Qumulo Care for any sort of question, whether it’s big or small. We use that feature a lot and we really love it,” said Meiser.