The problem with File Systems and Digital Assets
File system based media specialty servers continue to be in use in managing media such as digital images. The use of file systems to manage digital media such as images and video has led to specialty servers for different kinds of data. This results in data isolation and high systems admin and management costs. It also leads to scalability problems, high training costs, and complex support problems.
File systems with their limited name space and indexing mechanisms notoriously perform poorly under high load. They are also not noted for running with high availability. They are also expensive to develop, do not make it easy to maintain integrity of the content because of their lack of support for synchronized updates, and in general cannot be easily and centrally manage.
The Problem with Files: Searching/Scalability
File systems present several searching and scalability problems including:
- Manual file/directory naming. Problem: Inconsistent classification - lost/hard to find data
- Sequential file/directory searching. Problem: Waste time searching every directory and disk
- Limited search attributes. Problem: Waste time searching; can’t find; must reinvent
- No back office integration. Problem: mited searching, multiple copies of data, unsynchronized updates
The Problem with Files: Sharing/Security/Data Integrity
Files are also problematic when it comes to security and data integrity.
- Limited sharing (requires file shares). Problem: Content duplication, unsynchronized updates
- Inconsistent authentication (across servers) no audit trail, rollback, roles or views. Problem: Accidental file deletion/overwrites, malicious mischief
- No resource limits or record locking. Problem: Data can’t be shared, multiple copies of files
- Business rules ignored. Problem: Business chaos
The advantage of storing assets in a database
File system availability and performance do not approach the demands of 24 x 7 operations. With assets stored in a database, there is:
- Load balancing.
- Failover.
- Replication - distributed synchronization.
- Flexible indexing.
- Dynamic query optimization.
- Partitioning.
- Online incremental backups.
With a file system the risks include limited scalability, availability, and reliability.
Storing media in the database makes it easy to search and find any content, easy to develop new and extend existing media applications, easy to deploy with enterprise quality of service, and easy to manage the data.
If you enjoyed this post, make sure you subscribe to the RSS feed!
One large advantage, particularly with web applications, in filesystem storage tends to allow direct recall in a much more efficient manner. In a number of cases, it’s entirely reasonable to use your application server to manage the files on the filesystem, and then let your web server serve them directly off the filesystem back to end users, thereby freeing your app server for handling actual dynamic requests, rather than using threads to just serve static content.
Not always going to work, of course. If you need to control access to the files, for example, but for public sites that don’t need access control, the load reduction can be HUGE.
Actually it all depends HOW the content is being managed. Of course it might not always make sense to store your digital assets within the database for just end user stuff or if you are a webdeveloper who is doing a site where no one else is ever going to touch your code then you.
But think about a Content Management System, think about a Digital Asset Management, think about a worldwide organization where many contributors work on the web content. You want to use a central location for all your assets.
This is just one case were such a system would make sense. In my post above I mentioned many different scenarios as well.