Qumulo’s real-time metrics and performance increases the company’s productivity with a rich API and lower costs
After studying the market and carrying out POCs, Qumulo stood out for its performance, the quality to price ratio and, above all, the richness of its API.
Cognacq-Jay Image, one of Europe’s leading service providers in the media industry, helps broadcasters, telecom operators, OTT platforms and content owners manage their linear and on-demand workflows. They convert thousands of files for broadcast via television, Video on Demand (VOD) platforms and streaming services each day. The company has more than 500 clients and employs over 150 people at its two offices in France (Issy-Les-Moulineaux) and the United States (Los Angeles).
Scaling up storage infrastructure
Since its creation in 1996, Cognacq-Jay Image’s processing needs have continued to grow, and the company expects them to double before long. Most of this processing is carried out on its storage infrastructure. In practice, Cognacq-Jay Image does not store its customers’ files over time: they are transferred to its infrastructure via an in-house solution called SmartJog, then processed before being sent to the recipient. The company needs an average of 30 terabytes to manage its daily workload because, once processed, the files are deleted from its infrastructure.
“It is, above all, a work storage system,” explains Michel Desconnets of Cognacq-Jay Image’s Systems and Infrastructures Department. “In other words, our constraints are less about volume and more about access time and processing capacity.”
Faced with an increase in the video flows that required processing, the company decided to modernize its infrastructure by focusing on capacity.
“We require a volume of 100 terabytes in order to store all of the files from the previous day that are still being processed plus those that have not been dispatched yet,” says Desconnets. “The most important thing about our infrastructure is that it operates on a ‘just-in-time’ basis with daily deliveries of video streams.”
Gaining visibility to anticipate slowdowns
When the company decided to renew its existing infrastructure at the end of 2020, it was looking for a scalable solution with high read/write performance that was capable of providing very detailed metrics to facilitate monitoring and gain in proactivity.
“We wanted to increase read/write performance to 6 GB per second on the cluster and to be able to quickly identify the slightest source of problems,” explains Desconnets. “After studying the market and carrying out POCs, Qumulo stood out for its performance, the quality to price ratio and, above all, the richness of its API. All of the cluster’s functionalities are included in this API, which not only provides very detailed metrics, but also makes it very easy to integrate the cluster’s supervision into our administration console.”
In addition, unlike other solutions that provide ad hoc reports, Qumulo’s API retrieves metrics in real time. This is a feature that is greatly appreciated by Cognacq-Jay Image. The company processes dozens of transformations on the fly and sometimes has to manage thousands of files simultaneously of a very wide variety, including web videos which are generally very small in size.
“If you multiply a latency that goes from 5 milliseconds to 50 by the number of files processed simultaneously, it’s easy to understand why we have to monitor our infrastructure very closely and in real time in order to react to the slightest anomaly and anticipate bottlenecks,” says Desconnets.
In addition to latency metrics, the Qumulo API allows Cognacq-Jay Image to perform a real-time audit of everything that happens on its storage infrastructure. In particular, they like being able to see, at any time, who is doing what–whether it is a modification, a deletion or even a reading of a file.
Real-time management
Installed by HPE and Qumulo, the six 36TB nodes (150TB useful) went into production almost immediately: the video streams were simply directed to the new work storage platform. A year later, satisfied with the system, the infrastructure was extended with two new nodes to meet the company’s growing needs.
“In the end, it was the integration with our administration console that took the most time, but it was still very simple: a few scripts were enough to retrieve all the metrics we needed,” adds Desconnets.
