9 Story Media Group is an industry leading creator, producer and distributor of award-winning animated and live action content for young audiences around the world. Brown Bag Films is 9 Story’s animation studio, responsible for programs such as Daniel Tiger’s Neighbourhood, The Magic School Bus Rides Again and Doc McStuffins. Brown Bag has offices in Toronto, Dublin and Manchester.
Summing up the core values of Brown Bag Films, Chris Brown, Sr. Systems Engineer at the Toronto office, says, “Our business is children’s animation. We make shows that entertain and educate, that’s very important to us. The other thing to know about Brown Bag is we love animation!”
Revamping the Pipeline with QF2
Brown Bag brought Qumulo File Fabric (QF2) to its Toronto office as part of a strategy to revamp Brown Bag’s post-production pipeline with more modern practices. Chris says, “Traditionally, our production pipeline used NAS, but we did our post-production work on direct-attached storage. We used Avid Media Composer as our editing software along with its proprietary storage solution, but there are problems with that approach. Isolated storage encourages data redundancy, and it’s difficult to come up with a good strategy against data loss, so we wanted to bring in a shared-storage solution.”
“We investigated block storage but, to me, it was a step backwards. We wanted to be flexible and accessible. Then, we looked at quite a few file vendors, but I’d used QF2 before when I worked on a feature film a few years earlier and it was a great experience. My management liked the solution Qumulo proposed, so here we are. Now, we use QF2 in both our production and post-production workflows. We have eight QC104s at our production site.”
Small Files? No Problem!
A characteristic of the Brown Bag—Canada workflow is that it generates billions of small files. These are largely marker files and index files used by the Adobe Premiere and Avid editing software. Editors use marker files to indicate important points in time and to leave comments. Index files are used to quickly locate files in the editorial software’s database. Chris says, “We do have issues with our legacy file system, which slows down when there a lot of small files. We wanted to improve that when we took on our modernization project. We’ve been successful with QF2 and that was a big win for us.”
Post-Production with QF2
In post-production, all of Brown Bag’s editing is done on the QF2 cluster. Most of the animation is in HD. The company uses Avid Media Composer and Adobe Premiere as its editing software. There are six productions and each show has a senior editor and an assistant editor. A single editor works on a single stream and there are multiple editors working on different workstations, all accessing the QF2 cluster. Of course, the cluster is also busy with production workloads at the same time that the editors are accessing it.
Keeping Data Safe
Along with its production cluster, Brown Bag—Canada also has a disaster recovery (DR) site where they run QF2 on four HPE Apollo 4200 Gen 9 servers that hold about 450TB. Chris and his team decided on the HPE servers because the DR site has limited rack space, and the HPE Apollo 4200 has a 2U form factor optimized for density.
The two sites are connected by QF2’s continuous replication, which moves data where it’s needed, when it’s needed. Chris splits replication up into many parallel jobs to get the maximum use of the bandwidth and to efficiently perform the initial synchronizations. Activity varies, where some jobs are fairly static, such as legacy data that is read only. Other jobs, such as production and post-production work, change from minute to minute. He has a blackout period that lasts from 8 in the morning until 8 at night.
Chris’s team also takes snapshots hourly on all the production and post-production workflows. They keep the hourly snapshots for two weeks. They also take weekly snapshots, which they keep for three months.
QF2 gives administrators complete, real-time visibility into the file system. Chris says, “One of the reasons we wanted to use QF2 is because it gives you the ability to drill down and look at what’s happening, all the way to the file level. Traditional arrays don’t have that granularity.”
“With QF2, if there’s a problem with an edit workstation, you can pinpoint that, for example, it’s a bandwidth issue. That’s fantastic.”
“You can look at a client’s throughput and how many IOPS they’re using and understand what’s required for them to work on a particular production. You can see who’s using what storage and when and how. Being able to track that over time is very helpful. It makes my forecasting a lot easier. Any time there’s an issue, the first thing I do is look at the Analytics UI to see who’s doing what. It’s just a great tool.”
Qumulo Customer Care is another reason Brown Bag feels such a strong loyalty to Qumulo. Chris says, “I’ve been working in storage for a long time and, of all the vendors that we use, I feel like Qumulo is on this journey with us. We’re not just vendor and customer. We’re partners. Any time I have an issue or any time I have a question, the folks at Qumulo are all over it.”
“I have tons of examples of how my friends at Qumulo have saved my backside after a mistake I’ve made or something that I haven’t remembered to do, or if there’s an issue we have. Reaching out to them on my own Slack channel is instantaneous and the resolutions are very, very quick. It makes my job a lot easier and I sleep better at night. I’m happy to say that Qumulo has made me look very good to my management and I appreciate it.”