Storage Requirements for Video Recording

Regardless of the sector of the media and entertainment industry, the specifics of each carry specific requirements for transferring, storing, writing, and reading data. The gaming sector brings together the work of artists, graphic designers, developers, sound designers, and voice actors. The movie industry relies on creators, actors, technicians, or post-production specialists (i.e., editors, sound designers, etc.) and requires maintaining the above-mentioned aspects at the highest possible level. Thanks to the work of creators and the streaming platforms offer, which is increasingly popular with viewers, there is easy access to pre-recorded video files available to viewers anytime. Imagine going through a pandemic without streaming services – tough, huh? Cinemas also rely on digital copies of films as it’s easier to be accessed and played. The work of so many people at every stage of content creation – both games and movies or videos, carries the risk of data loss in transmission or degradation while storing. So it would be a disaster to be forced to repeat the entire process in case of any data corruption or loss.

CCTV instead MTV

There is another sector of video recording worth mentioning but not related to journalism or art – security. CCTV carries specific requirements in this regard. Recording takes place almost non-stop, and the size of saved files often exceeds those for movies or journalistic recordings. Data has to be available long after it was recorded and saved for later access. Also, data corruption can be a more significant problem here. Stored data is exposed to corruption over time, which is unacceptable as security data may be used as proof or evidence in court.

So what’s important to make data recording and storing a smooth, hassle-free process? What level of the technical facility should be delivered to make the job a simple task rather than an inconvenience? Let’s find out!

Efficiency, Safety, and Data Accessibility

There are three factors – accessibility, efficiency, and safety – which are extremely important to consider if we look at data storage from a video recording perspective. 

Let’s start with the most significant aspect – data safety. In streaming and distribution (music and film industry), the data is being kept for a long time to be accessed anytime in the future. That’s why system scalability is an important aspect. You don’t know how much space the next project will require, and thus you should always be able to increase it. Data corruption or security breaches (i.e., ransomware attacks cutting us off from the data or causing downtime) are things to avoid in the first place. Imagine losing all security recordings from a bank or making them publicly accessible. The consequences could be terrifying. That’s why making snapshots of the data and an easy restoration process with Open-E JovianDSS is something we used to highlight so often. Regarding the data corruption mentioned earlier – as Open-E JovianDSS is based on the ZFS system, it allows self-healing and scrubbing. Sounds good, huh? But that’s not all.

Let’s not forget about natural disasters or thefts. Using only on-site data protection, the storage facility may get destroyed by fire, hurricanes, or floods. There’s a possibility that a thief knows how the storage units are secured and can easily access them unnoticed and steal the hardware. 

You can now spot the situation when off-site data protection with Open-E JovianDSS comes to the rescue. It prevents you from losing data in case of the above-mentioned circumstances as it allows storing and accessing it externally via ethernet. Other advantages are:

• entirely integrated and speedy data recovery
• no need for extra backup agents
• low RPO and RTO lowered to a minimum regarding data loss and recovery 
• high customization level.

It should also be possible to upload a video to storage quickly after it’s prepared or edited. As mentioned before – lots of people are involved in production. Very rarely, almost never, it’s one person. The files tend to be large and accessed by multiple devices, no matter if it’s a production team member or just an end-user. Journalists and reporters often upload their video materials just a few minutes before it’s broadcasted live on TV. Hence, high performance on the write operations is pretty significant. It can be assured by the high bandwidth of network adapters, especially in the case of videos recorded with high-resolution cameras. We can’t forget that high-resolution recording makes the video files big, sometimes huge. Therefore, it’s beneficial if the storage environment allows us to achieve both good write performance and high capacity. Quite often, copies of a file are stored in one place so the editor can prepare its variations. As you can see, accessibility and efficiency play a colossal role here.

There are a couple of other functionalities of Open-E JovianDSS that will make storing videos more efficient and safe. It allows you to configure your backup to ensure data redundancy. What we recommend:

• RAID-Z2 or Z3 for security and better capacity 
• 2-way or 3-way mirrors if performance is critical and with systems with too few disks to create few RAID-Z2 or RAID-Z3 data groups
• MPIO (for SAN) or Bonding (for NAS) for better performance and connection redundancy

The Hardware You May Need

Ok, so once we showed you the best practices and approaches to make your video storage safe and efficient, let’s focus on details – the hardware. Based on our experience in such an environment, we recommend using particular equipment such as:

• Chassis: 
  • a large case, e.g. 4U for 3.5″ drives, top-loader JBOD.

• CPU: 
  • Intel Xeon Silver series high frequency or AMD equivalent should be enough.

• RAM: 
  • 64GB or 128GB in most cases is perfectly adequate.

Except that, we suggest implementing a couple of well-functioning practices, like:

• Data groups: 
  • High-capacity NL SAS disk drives (for Shared Storage Cluster) or some high-capacity SATA disk drives (for Single Node). SATA disks are only inside the server and not in external JBOD.
• Storage controller: 
  • It is good practice to have a disk controller with an additional external port to allow storage capacity expansion.

• Write log: 
  • When the recorder stores data locally and then replicates that data to Open-E JovianDSS storage, it is possible to change the txg_timeout setting to 1 second and sync to standard in volumes and then not use a dedicated Write Log at all.

In this case, the read cache is not that important as long as you are not streaming. Therefore, there are no special requirements.

To clarify the key hardware investments (and the not-so-really ones), we picture this in the form of a graph:

We Want to Make a Difference

Recently we’ve partnered with a Czech non-government organization Post Bellum that seeks out videos and records recollections of historical events from the 20th and 21st centuries. That was an amazing chance for Open-E to support passionate journalists with our product and its features, as well as assist in building the hardware structure for better storage capabilities at Post Bellum. As you can see, what we described in this article is taken directly from our everyday work where we learn more and more about how to build different storage implementations for various companies and their specific requirements. Thus, we understand the media, entertainment, and surveillance market needs. We’re sure this bunch of advice and solutions can apply to your business to make your video storage system safe, efficient, and easy to access.

Check out our Case Study with Abacus Electric for Post Bellum!

• CCTVgaming