Thread: Storage and archival options

Reply to Thread
Page 1 of 2 12 LastLast
Results 1 to 10 of 14
  1. #1 Storage and archival options 
    Senior Member Christoffer Glans's Avatar
    Join Date
    Jun 2007
    Location
    Stockholm, Sweden
    Posts
    4,370
    By now, it's no mystery that Komodo will require more disk space for the material that we shoot. For anyone who views Komodo as their new A-cam rather than a crash cam, this will probably be the biggest challenge going forward.

    What might be good options going forward? Good drives that are cheap, auto-backup type drives? Archival options for RAW material and so on?
    Even though this is always something important to discuss, I believe it to be more so going forward with Komodo as an A-Cam.

    Thoughts?
    "Using any digital cinema camera today is like sending your 35mm rolls to a standard lab. -Using a Red is like owning a dark room."
    Red Weapon 6K #00600
    Reply With Quote  
     

  2. #2  
    Storage is cheaper now than it was in the past. Anyone already shooting with other 4K+ cameras already knows what they’re getting into. So same as before, reasonably sized RAIDs/ SAN systems for online and near-line storage. Bare HDDs for bulk storage, supplement with LTO tape for critical data if or as budget allows. That’s pretty much our current strategy. Good backups are a lifesaver. And eventually everything needs to be moved to long term storage if you want to keep it. Backup and archival is more a strategy than about a specific technology. Whatever methodology you choose, plan to revise that every now and then. Plan to revisit those backups and archives every so often and migrate them forward to newer tech. My oldest backups are on LTO-4 tape and newest on a combination of HDDs and LTO-6/ LTO-7 and I will pull them forward at some point here. Probably when moving to LTO-9 or other archival solution that I decide on in the next year or three.

    I don’t shoot anywhere near as much these days as I did in the RED One and EPIC times, but I stil produce just as much or more data with all that I do. Camera files have grown, other image and app data has grown... This past year, 10~12 TB HDDs have been the sweet spot for bulk storage and my RAIDs/ SAN systems. I’m all SSD storage for onboard and direct-attach on my workstations for performance and increased reliability.
    - Jeff Kilgroe
    - Applied Visual Technologies, LLC | RojoMojo
    - Just me and my 8K Monstro VV kicking ass.
    Reply With Quote  
     

  3. #3  
    Senior Member Christoffer Glans's Avatar
    Join Date
    Jun 2007
    Location
    Stockholm, Sweden
    Posts
    4,370
    Yes, this is what I mean. A thread about what solutions that are good, both strategies and also what products that are good. It's a topic that I think would be good for those planning workflows.

    What do people use? What strategies are good etc.?
    "Using any digital cinema camera today is like sending your 35mm rolls to a standard lab. -Using a Red is like owning a dark room."
    Red Weapon 6K #00600
    Reply With Quote  
     

  4. #4  
    Senior Member
    Join Date
    Nov 2011
    Location
    Burbank, CA
    Posts
    741
    My story is the same as a lot of others. Started way back in 2006 with some Western Digital externals. Moved up to a 4 bay RAID 5, then got a second RAID array to back up my first one.

    Right now I've got a Synology 1819+ in RAID 6 with 12TB disks, which back up nightly to a local RAID array and over the internet to a Synology 420j at my parents' house. This gives me triplicate backup with redundancy on all three devices, with about 700-900MB/s speeds out of the Syno. I would recommend a DS420 or 920 to begin storing things, but if you want to edit off of it, you'll want at least 8 drives with a 10Gbe network. You need the 8 drives to be able to saturate the bandwidth of the 10 gig connection.

    You could go with T5 SSDs, but that's a single point of failure, though quite unlucky. You could go with an SSD or NVMe RAID, which would provide extraordinary performance, but might be a bit overkill for an owner/operator, and certainly isn't the most affordable option.

    Curious to hear what everyone else's setups are.
    REDucated June 2012

    Scarlet-X #2098
    Raven #61
    Komodo #234

    Hi YouTube!
    Reply With Quote  
     

  5. #5  
    Digital FX Greg M's Avatar
    Join Date
    Dec 2006
    Location
    Louisiana
    Posts
    5,110
    LTO is the only real option for archiving footage. Storing on spinning drives is an accident waiting to happen.
    We have been using LTO for over 11 years now, and it has changed our life. We have over 800TB of footage on tape, and being able to search a database and restore a project from years ago is a game changer. We have nightly backups of the entire SAN happening each day and separately we archive footage and projects to LTO. Far cheaper and more manageable than drives, and we add a small archive fee to every project to offset cost. Plus bill for restores from our archives.

    and I'll just add that during the initial COVID shut down, life would have been miserable with any other solution. We edited new commercial campaigns during shutdown for two clients that were completely assembled from past campaigns. The ability to quickly find and restore those assets from LTO made that possible.

    The upfront cost may seem high, but its really not...and once you make the change, you will only regret not doing it sooner.
    www.digitalfx.tv

    Twitter
    Instagram

    RED One #83
    RED CF Monstro 8K VV
    RED CF Weapon 8K S35 Helium
    RED CF Weapon 8K S35 Helium
    Reply With Quote  
     

  6. #6  
    My only regret is that LTO-9 is not available NOW!
    Michael Tiemann, Chapel Hill NC

    "Dream so big you can share!"
    Reply With Quote  
     

  7. #7  
    Member Mark A. Jaeger's Avatar
    Join Date
    Jul 2019
    Location
    SE Washington
    Posts
    74
    Michael T - do you have a recommended vendor for the LTO backup solution?
    Reply With Quote  
     

  8. #8  
    Quote Originally Posted by Mark A. Jaeger View Post
    Michael T - do you have a recommended vendor for the LTO backup solution?
    Not really. I have some LTO-5 and LTO-6 drives, but am waiting for LTO-9 to refresh everything.
    Michael Tiemann, Chapel Hill NC

    "Dream so big you can share!"
    Reply With Quote  
     

  9. #9  
    Digital FX Greg M's Avatar
    Join Date
    Dec 2006
    Location
    Louisiana
    Posts
    5,110
    Quote Originally Posted by Mark A. Jaeger View Post
    Michael T - do you have a recommended vendor for the LTO backup solution?
    Id recommend Bob Roose at TM Television
    www.digitalfx.tv

    Twitter
    Instagram

    RED One #83
    RED CF Monstro 8K VV
    RED CF Weapon 8K S35 Helium
    RED CF Weapon 8K S35 Helium
    Reply With Quote  
     

  10. #10  
    Senior Member andrewhake's Avatar
    Join Date
    Oct 2009
    Posts
    660
    It really depends more on if you are an individual managing your own data or part of a small or large studio. If it's the latter then it really should be a dedicated person to handling it all. If you are an individual a good modern machine with TB3, a good TB3 storage array, and a good online backup solution with a fast internet connection, is infinitely easier and more accessible than LTO options.

    I have an array for active projects and other that acts as an archive. Both get backed up to Backblaze over a 1Gb/1Gb connection for $55/year. I personally use the G-Tech Shuttle TB3 arrays and have yet to have any issues with them with them on 24/7. I like having 2 separate arrays so if one goes down for whatever reason there is another ready to go. This is just a personal non-professional solution that I use for my own video and vfx projects. If you are working on projects with data security issues then this might not be viable.

    Obviously for a small or larger studio with multiple people you need to start looking at a dedicated SAN setup and someone to manage it and might not be able to utilize off the shelf backup solutions for security reasons. If you never check your LTO archives or do anything with them, it is much more useful to just keep an online archive and pay for the storage that you need to do that. LTO definitely has it's uses but having an online archive is much more useful if you can afford it. The setup I use professionally for work deals in PB not TB and 100Gb connections, so that is an entirely different solution with an entire team dedicated to it.
    Last edited by andrewhake; 08-15-2020 at 04:10 PM.
    Reply With Quote  
     

Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts