Media assets push the boundary of storage systems out there, be it capacity or performance. Media asset management solution is often the best friend a media asset manager can deploy to manage storage at scale.
However navigating the world of media asset management (MAM) storage can be overwhelming with ever increasing options especially in the cloud.
Not anymore, thanks to this guide. We explain the variety of storage options available for MAMs on-premises as well as in the cloud, to ease the task of selecting the best storage solution for your workflow.
Media Asset Management storage is optimized for two primary scenarios today:
- Meeting the needs of post-production teams who need high-performance storage to edit 4K or higher resolution videos clips
- Archiving videos especially raw or native camera cards at scale.
Before we delve in, let’s discuss the challenges that post-production teams face, so that we can better understand where available storage options fit into solving these problems.
Table of Contents
Post-production storage challenges
Archive storage challenges
Archive or perish is the refrain that reverberates through the corridors of media managers. Perhaps no other component of Media Asset Management is more desired by media managers or media operations teams than storage & archive management.
Why are archiving needs such a hot topic for media asset managers?
As we will explore in this in-depth article, native video assets with their expanding resolutions (SD -> HD -> 4K -> 8K -> 16K coming soon) can occupy Terabytes (TB) of storage for as little as a 30-minute video footage recorded by a modern video camera.
Without comprehensive storage & archive management at scale, media managers will have a hard time with:
- Ingesting new and existing video content without constantly buying more disks
- Overrun IT budgets when using cloud storage such as AWS S3
- Locating clips in video archives for their editing teams which can take days even for the most diligent media manager
- Sharing video projects with freelancers or remote editors which can become an endless stream of uploads with no end in sight for video post-production teams
In trying to combat these, you will need to evaluate available the storage management capabilities of Media Asset Management (MAM) systems to understand which best addresses your team’s needs.
Here are some points to consider when choosing the best post-production video storage in the context of MAM systems.
Choosing the Best Post Production Video Storage
What are the best types of storage for editing media assets today?
Irrespective of where you edit your videos – on-premises or via proxies in the cloud, there are choices & trade offs you will need to be aware of.
For an editor working on a final conform, there’s no worse nightmare than finding that the high-resolution video footage you need is either unavailable, or the underlying storage device performance is too slow, or unsuitable for editing operations like scrubbing or cutting.
When considering types of storage needed for post-production assets, the following 3 factors are most important:
Performance
Is your team simultaneously editing multiple 4K footage at speeds of 50 Mbps or more? Do they complain of lag in their editing suites (Avid Media Composer, Adobe Premiere Pro, etc)? Does the final conform take more than3 hours to render in a transcoder like Adobe Media Encoder?
If you answered a ‘yes’ to any of these, it means that your storage performance is not optimized for video workflows and you need an upgrade.
Cost
Did your tech-savvy editor convince you to invest in high-speed Fiber channel (FC)-based Storage Area Network (SAN)?
Are you finding that as your storage volume increases you’re face with an unanticipated 5-6 digit storage bill? Are you worried about being able to justify the incremental cost of on-premises storage 3 years from now?
If yes, the good news is you are not alone. Every media manager worth his salt is probably grappling with increasing storage costs – even with research showing that the cost of storing 1GB of data has reduced by 99.99% from $2 million in 1960 to $0.02 in 2017.
On-premises vs cloud
On-premise storage seems a lot cheaper per GB than cloud storage when you consider storing TB or PB in the cloud. However, there are hidden costs associated with on-premise storage that often tip the scale towards cloud storage.
The usual suspects that drive down the ROI of on-premises storage include:
- Cost of Disaster Recovery (DR) & backups
- Data center costs
- IT management overhead
- Lack of availability due to inevitable downtimes
- Lack of remote access without compromising security
- Security. Increasing tales of ransomware attacks sends shivers down the spine of veteran media managers. What impact will a ransomware attacker encrypting your critical storage on-premises have on your business?
Now that you have an understanding of the factors affecting your choice of storage, let’s review the best options for on-premises vs. cloud storage and their pros & cons:
On-premises media storage options
Direct Attached Storage (DAS)
This is the simplest and the most common option especially for small editing teams. This can take the form of internal storage devices such as NVMEs, multiple external USB/Thunderbolt drives or even a fast RAID drive enclosure directly attached to an editing workstation.
Pros
- Consumer-grade devices offer great performance at an affordable price. For instance the NVME.2 internal SSD in a Macbook Pro M1 Max laptop can clock ~ 5 Gbps reads & writes – suitably fast for video editing
Cons
- Can’t offer storage at scale. Beyond a few dozen devices reaching 6-8 TBs, it gets difficult to keep chaining more devices
- Can’t be shared easily with other editors without using slower file streaming protocols like SMB or NFS
- Poor DR. Often outside the purview of a centralized IT backup, the editors are left to fight disk crashes
- Poor archiving and retrieval. It is not uncommon to find tons of full external drives sitting in a editor’s cupboard with no content search capabilities, making it impossible to reuse content
- Poor security. Despite virus or malware scanners, these are susceptible to ransomware attacks as amply demonstrated in recent times.
Storage Area Network (SAN)
Storage Area Networks (SANs) are high-speed networks or subnetworks that are specialized for creating network access to storage drives. These SANs consist of interconnected groups of servers and storage devices benefitting from high-speed architecture.
SANs have been the workhorses for media shops that need fast video editing storage solutions.
Pros
- SAN Fiber Channel based host bus adapters (HBA) in edit stations offer 16Gbps+ storage access speeds – a must have for large editing teams that need a shared storage for links to footage or rendering out large video exports or animation sequences
- It’s easier to scale up by adding more storage units to the network when you fill up existing volumes
Cons
- Cost. The extra HBA cards needed to add to, and maintain edit stations and drivers can be a huge drain on IT budgets
- Security problems. Even with virus or malware scanners, SANs are still susceptible to ransomware attacks once hackers gains access into corporate networks – so this is often their primary target
Network Attached Storage (NAS)
Network-attached storage is a specialized file storage system that is provided through a computer network, allowing heterogeneous devices and multiple users to access and retrieve data.
Pros
- NASs have evolved with the advent of 10Gbps ethernet switches & networks. A fast 10 Gbps card in the edit station gives your editors a headstart when scrubbing through multiple 8K clips in parallel
- Like SANs, NASs are easy to scale. As long as the NAS enclosure permits you to add more disks, you can increase capacity fairly easily
- Cost-effective. NASs are more affordable than SANs especially when using standard 1 Gbps wired connections
Cons
- Mid to large teams with multiple editors will need 10 Gbps ethernet cards on their workstations to scale. This can significantly increase costs making them comparable to SANs
- The ubiquitous SMB protocol used for file transfers with NAS can be slow compared to FC in SANs
Cloud Storage
Every major cloud provider like Amazon Web Services (AWS), Google Cloud or Azure Cloud offers a number of storage options for storing media files. Only a handful of them are suitable for editing in the cloud:
Cloud Block Storage
Like their on-premise DAS counterpart, these are block level storage devices that are optimized for access speed. Examples include AWS EBS or Elastic Block Storage.
Pros
- Offers 1 Gbps+ IOPs for common editing scenarios in the cloud. For example, remote editors can use PCoIP apps on their on-premises desktop to remotely access a cloud-based virtual desktop with an attached cloud block storage
- Superb DR. These are often backed up using snapshots in 2-3 availability zones as well as multiple regions in the cloud
- Better security management. Because these sit behind a Virtual Private Cloud (VPC), they provide better- security by allowing for a centralized security management with regular software updates
Cons
- Super expensive. An AWS EBS 5TB volume can cost over $600/month! Compared to a DAS this may seem outrageous but let’s not forget the overall ROI of on-premises DAS decreases when you consider the factors mentioned earlier
- Limited scalability. For example, AWS EBS can’t expand beyond 16TB without using complex software RAID configuration. They also need to be provisioned as fixed size volume and manually resized by the administrators with significant downtime involved
- Limited shareability. These cannot be shared with other edit stations in the cloud
Cloud NAS (Network-attached Storage)
Using NFS, vendors like AWS now offer storage solutions like EFS (Elastic File Storage) that can scale to unlimited sizes! But before you get too excited, here are the pros and cons of this storage option:
Pros
- Highly available and durable. For example AWS EFS is designed for 99.999999999 percent (eleven 9s!) of durability and up to 99.99 percent (four 9s) of availability.
- Cost-effective. Unlike block storage, you add file data without provisioning a cloud storage in advance. This way, you only pay for what you use, not for a provisioned storage pool that you only use a fraction of.
- Almost no limit on data storage. Storage size can be scaled up transparently to PBytes without requiring additional storage management
Cons
- Slow. For media operations, Cloud NAS can have high latency that makes it impossible to implement a post-production workflow such as video editing. Unlike their on-premise counterpart, these can not sustain high-bandwidth, low-latency connections from edit workstations.
- Weak consistency model: Multiple applications accessing the same file or folder can run into inconsistency with asynchronously executing storage operations. For example, an application might rename a file, but the change may not be immediately visible to another application.
- On-premise access often uses a protocol like NFS, and ultimately high-latency WAN connection can lead to time outs, write errors, and caching issues.
Cloud Object Storage
Cloud object storage is a system for storing unstructured data in the cloud. Object storage is a great fit for the cloud because it can easily scale from Gigabytes to multiple Petabytes to support unlimited data growth in the cloud.
Use cases include archiving media content in the cloud, streaming video proxies from the cloud into on-premise edit stations.
Choosing the Best Archive Storage
What are the best types of storage for archiving media assets today?
Given the relentless growth of video content in driving modern digital experiences for marketers, it is obvious that the cost of archiving video has skyrocketed for media operations teams.
Only a few years ago, HD video was the latest craze. However modern-day video resolutions (e.g 4K and 8K), and capture technology likeAR/VR with 48+ cameras can generate Terabytes of footage from just one video shoot!
Similar to post-production storage, as a media manager, you will need to make choices and consider trade-offs irrespective of where you plan to archive your videos – on-premises or in the cloud.
But no matter what, there is no worse nightmare for a media manager than finding out that archived footage is corrupt only when the video editing team asks for it.
When it comes to types of archive storage for media assets, the following 3 factors must be top of mind for you:
Data Preservation
From on-premises NAS, to LTO tapes to cloud archive storage, there are choices to make. But at the end of the day data preservation is the primary objective of all media managers.
What good is an archive with poor durability guarantees? Think carefully before optimizing for costs when you decide to implement a data preservation strategy for your video archives. Why?
Well, imagine your video team trying to create a tribute to Queen Elizabeth in 2022. In trying to find footage from1961 when she visited India, your team finds that the magnetic tapes have degraded and are unusable!
Obviously, the footage can’t be re-shot. This is why digital preservation is more important than anything else when it comes to maintaining video archives.
Tiers
As a media manager, it is your responsibility to ensure that new footage needed in the short term can be retrieved from archives quickly (in minutes, not hours if possible).
However, dated, historical footage can take hours or even days to retrieve, and that is often fine with most post-production teams.
However, planning and creating a retrieval schedule can set realistic expectations for the post-production team, as they execute restore tasks in order of priority without frustration over slow footage retrieval delaying editing.
This is where the role of archive tiers becomes key.
Archive tiers allow media managers to split a large trove of media assets into multiple categories with different archive & restore speeds. On a speed spectrum, fastest to restore would be SSD/NVMe storage, with the slowest being off site or cloud-based tape storage.
Costs
Given a choice, post-production teams would probably prefer to put all their video archives into expensive SSD storage for instant restores.
But Santa doesn’t create budgets, so the cost would be prohibitive. For example, the annual cost of 100TB of SSD based AWS EBS Cloud Storage could exceed $150K per year!
Restoring files from especially cloud storage incurs EGRESS or Data Transfer Out fees that can be substantial. The cost of on-premises LTO tapes may seem like a bargain until you have to deal with Disaster Recovery (DR) requirements.
What Type of Storage is Best for Archival Purposes?
To cut to the chase, beyond 2022, Cloud Object Storage with multiple tiers should be the only option a media manager should consider.
As discussed in the post-production section above where we evaluated the pros and cons of various types of media asset storages, Object Storage solutions (like AWS S3) offer the best performance against cost tradeoffs in the marketplace today.
LTO tape-based archive strategy is obsolete now, for the simple reason we mentioned above – digital preservation. Providers like AWS are able to offer unimaginable guarantees such as 99.999999999% of data durability for AWS S3 Glacier Deep Archive tier.
A media manager looking to recreate this on-premises will require a level of expertise and scale that is simply out of reach for them.
To better understand the cost of cloud archive tiers, you can check out a free cost calculator like this to estimate the costs involved. Keep in mind to not fall for the trap of comparing the cost with an inexpensive LTO tape – as tapes are the building blocks used by even the likes of AWS to offer high durability archives.
This requires AWS to replicate the archives in multiple availability zones or data centers with multiple redundancies in power supplies, network switches, routers, data cables etc.
A Robust MAM System Simplifies Post Production Storage and Archiving
Now that the various types of storage have been clearly laid out, let us understand how modern Media Asset Management (MAM) systems play an outsized role in your storage management strategy.
Transfer Speed
- Transfer speed is of utmost importance, especially when managing large video workflows from multiple remote-based locations. Therefore, it is important to choose a MAM solution that implements a strategy for optimizing file transfer speed from its video archives, specifically for large files. For example, Evolphin Zoom’s Cloud Sync Agent desktop application enables 10x faster transfers for large media files by transferring files in small chunks.
Search
- You are never going to wade through restoring endless tapes just to locate a specific clip within a footage from 1961, thanks to the metadata-driven search by MAMs like Evolphin Zoom. These systems will index all your essential metadata terms even within a video timeline so you can find any asset or a sub-clip from the archive in seconds!
Proxies
- Video proxies generated by your MAM allow your post-production team to start editing while waiting for large high-resolution 4K footage to restore from a slower but cheaper archive tier like AWS Glacier. And once the high-res videos have been restored from the archive, the MAM’s editing plugins like Adobe Premiere Pro plugin can automatically switch the links from proxy to high-res video for final conform. This level of efficiency helps you save archiving costs without compromising your video production workflow.
Tier Management
- Archive tiers are key to managing your video archives cost effectively. But this creates a need to keep track of files that are split between 4-6 different archive tiers. Clearly, Excel or Google spreadsheets won’t cut it for tracking when dealing with millions of assets. This is where the automated archive management features offered by modem MAM systems come in. As long as you remember your key metadata terms, the MAM will automatically move assets between tiers to quickly get the restores started when the need arises.
- More importantly, if assets have not been accessed in a while, a robust MAM uses automated lifecycle rules to move the assets from higher priced tiers to lower-price tiers. The algorithms used by modem MAMs combined with the archive tiers offered by likes of AWS make it possible to have petabytes of assets in the cloud without breaking the bank.
- Scalability: Every media manager is concerned with scaling their archives from a technical and cost perspective. MAMs like Evolphin Zoom enable you to design a customized cost plan that works for your organization’s needs. For instance, you can keep 10% of new assets in a faster, costlier warm storage tier such as AWS S3 Standard and move the remaining 90% into a cold, less expensive archive tier such as AWS Deep Glacier.
Deduplication
- Front-end deduplication works at the point of ingesting new files into your MAM system. It compares new file versions with the original and selectively ingests just the data that differs from the original. This process reduces your storage needs and significantly cuts your storage costs.
From these, it is evident that your capacity to efficiently scale your digital video storage and archiving operations heavily depends on the capabilities of the MAM system that you choose.
It is important that you undertake an evaluation to define your archiving needs and ensure that your prospective MAM system fits into those needs.
Evolphin Zoom Is the Trusted Media Asset Management System for Digital Video Archiving at Scale
The 114-year old Italian football club, Inter Milan FC relied on Evolphin Zoom MAM to handle their entire digital video archiving and storage operations. Beyond managing big video workflows, InterMilan was able to leverage Evolphin Zoom’s capacity to ingest, retrieve, and collaborate effectively with AI assistance.
If, like InterMilan, you want to scale your digital video storage and archiving operations effectively enough to optimize preservation and retrieval of your video assets, Evolphin Zoom is your best choice.
Investing in Evolphin Zoom MAM gives you access to the best in class media asset management system that scales effortlessly to meet your needs.
You also get migration, technical set up, ongoing technical support and consultations included in your purchase so that you’re never left without support. If you’re ready to speak to someone about your MAM requirements, contact us today.