Any scientific investigation produces a large amount of data, which often includes high-resolution digital images generated by microscopes and other imaging technologies. In this blog post, I want to talk about one project that’s going on at the Scientific Computing and Imaging Institute (SCI), a research institute at the University of Utah.
Researchers at SCI are creating the retinal connectome of rabbits as a way to understand human retinas. The retina is a thin, delicate layer of tissue lining the back of the eye that captures light like film or digital sensors in a camera. But the retina is also an incredibly complex network of hundreds of millions cells that process light, converting it into electrical signals that are sent to the brain and used to create the images we see.
A connectome is a map of all the brain, spinal and retinal connections that exist in an animal's body. Understanding the pathways of this network, and how they are rewired by aging and disease, is helpful in trying to save and restore vision. If you are going to fix cells in the retina, you have to know how they communicate.
Creating connectomes requires high-speed automated imaging, automated computational map building and massive storage. A single 3D connectome map can require more storage space than 100 desktop computers. For the retinal study, the team used images from an electron microscope that scanned thin slices of the retina, with the end goal of stacking the images up into a high-resolution 3D volume. Each slice was about 200MB of file storage. The entire data set was 20TB, but with backups and other temporary files, it took up about 50TB of storage.
Because building a connectome model is so compute and memory intensive, the storage system needs to be able to feed that pipeline with enough data to keep it filled. The project started off using a different file storage system than QF2, and the researchers found that getting data from disk to the compute resources was an incredible bottleneck. They were frustrated because the complex processing itself took so long that they couldn't also afford the extra time it took to load data into memory. The researchers estimated it would take four months to build the model, and they wanted to shorten that time from months to days. They also wanted to be able to manipulate the 3D volume in real time once it was built.
Coincidentally, the SCI IT team was evaluating QF2 using four QC24 nodes, and they thought that the rabbit retina project would be a good test, so long as they ran it while simulating the normal conditions at SCI, which meant 100 to 200 NFS sessions running concurrently throughout the day. By running the virtual sessions as well as the retina project, the IT team knew they’d get a good sense of how QF2 would work with the entire organization.
They found that, with QF2, they reduced the time to produce the 3D volume from four months to 11 days, with the system under full load. The researchers can rotate through the model, slice it, dice it, whatever they want.
Get the full story of the work at SCI
In case you’re curious, here’s what a slice of a dyed rabbit retina looks like:
If you’re involved with clinical or research imaging, then you have massive data sets that you’ll want to work with in real-time. If you’re a doctor, then you need to get patient information as soon as you can. If you’re a researcher, you want to get your results published as soon as you can. Not only does your storage system needs great performance but it needs to scale easily because high-resolution images are large and there will be a lot of them. If you’re a system administrator for a research or clinical organization, you’re also going to care manage your file storage system. Real-time visibility and accurate information about how the storage is being used will make your job a lot easier.
Contact Qumulo and find out how QF2 can fit into your research and clinical imaging workflows.
If you’re interested in learning more about how the QF2 architecture can save you money while giving you capacity and scalability, read the Qumulo File Fabric Technical Overview.
Keith is building highly successful teams that architect and build universal-scale storage systems for media & entertainment, life sciences, oil & gas, high performance computing and general purpose workloads
We are always looking for new challenges in enterprise storage. Drop us a line and we will be in touch.
Enter a search term below