R3D data manager???

[Chris Parker]

anyone used this? thoughts? opinions?

thanks

[Dylan Reeve]

I see Adam Wilt used it with his 'Three Three Letter Cameras' test - he made no comment on it though.

[Chris Parker]

oh yes.....here 's the link...

http://www.r3ddata.com/

[Chris Gearhart]

I purchased it this morning but haven't gotten a reply with the download information. I'll let you know how it looks.

-Chris

[Conrad Hunziker]

We have had several beta-testers, including a couple of prominent members of the pro video coalition. We have received feedback from them and incorporated it into our product.

In addition, we have several features that I think will make this a valuable tool, including quicktime renderings and FCP importing with scene/take info. Those will hopefully come out in a few weeks, and will be a free upgrade to all users.

[Etienne Caron]

Photocon,

Can you use multiple CF reader in firewire daisy-chain with your software. (http://www.lexar.com/readers/pro_reader.html)

Thanks

[Cüneyt Kaya]

what is the difference to aasync...aasync is free

[Conrad Hunziker]

Can you use multiple CF reader in firewire daisy-chain with your software. (http://www.lexar.com/readers/pro_reader.html)


The only requirement is a mounted file system. As long as your computer can see each card, so too can our program.

If you are wondering if you can do 2 copy operations at once, I think you'll find its not really feasible. Since you are pushing so much data around, your bottleneck is always the pipe you are using - firewire, eSATA, etc. So making 2 copy operations at once will take about 2.1 times longer than doing a single card separately. (The extra .1 is for bus overhead) For example, if it takes you 5 minutes to offload one CF card one at a time, it would take you 11 minutes to offload two cards at the same time. So its actually faster to go one at a time.

However, if you are doing card to card transfers, then there is no issue.

[Conrad Hunziker]

what is the difference to aasync...aasync is free

Aasync uses a sftp overhead, even with local copies. So it adds about 10% to the workload of the copy process, and would take about 10% longer.

In addition, aasync doesnt provide any sort of accountability or audit trail. Ive already run into a situation where a bond company required this from a feature I shot.

In addition, aasync doesnt provide any method for determining if a shot has already been copied or not. It simply copies files, which means that if the RED happens to create a take with the exact same file name, it will overwrite or discard your data - meaning you lost a take! And yes, I have already had this situation occur with the red, where it mistakenly went back a roll number, and there were matching filenames with different shots. R3D Data Manager will create a new folder automatically by default, unless you select a different option.

And then theres the future upgrades. We are already working on version 3, which will include quicktime rendering and scene/take info for import into FCP. In fact, attached is a screen shot of the quicktime renders window, in its current incarnation.

http://www.reduser.net/forum/uploaded/8155_1205989414.jpg

And these future upgrades are free for all users. So while aasync may be a good utility for remote copies, Id rather have something RED specific that can handle the finicky nature of our business.

[Dylan Reeve]

I've been using Rsync on OS X. It's a good option for the copy, but does lack and more advanced features.

Any plans for a Windows version Photocon?

[Anthony Gratl]

We have had several beta-testers, including a couple of prominent members of the pro video coalition. We have received feedback from them and incorporated it into our product.

In addition, we have several features that I think will make this a valuable tool, including quicktime renderings and FCP importing with scene/take info. Those will hopefully come out in a few weeks, and will be a free upgrade to all users.

One of those features is that you donate $5 to help preserve forests!!! Amazing. I love you guys! We need every company and group doing this! Thank you.

[Etienne Caron]

However, if you are doing card to card transfers, then there is no issue.

Exactly, its for "batch" transfers...

In the "color space" section, do you think in the future we could put the .LOOK file create in the Redone (or Redcine) ?

How long does it take to process 4K R3D to ProRes 422 HQ on a 2.5 GHz Macbook pro in your software for exemple ?


thanks

[Conrad Hunziker]

I've been using Rsync on OS X. It's a good option for the copy, but does lack and more advanced features.

Any plans for a Windows version Photocon?

As of right now, we are feeling the waters for windows support. One of the big issues is that there isnt very good red support for windows, for some of the more advanced features - such as quicktime renderings. RedCine is only supported for mac right now, so there wouldnt be a way to run this on windows yet.

So the short answer is that we may do a more 'dumbed down' version for windows, that wouldnt have those additional features until its supported by the big guy.

[Conrad Hunziker]

Exactly, its for "batch" transfers...

I dont mean to get picky here, but a batch transfer wouldnt work, and would have the same limitations as being slower than if you transferred one at a time.

Right now, the software is setup to handle one source (Red CF card, Red Drive, or a previously copied R3D folder on any medium) to one or more destinations. I really feel that multiple sources at once would be slower and not worth the time.

However, it does bring up a great idea - multiple sources sequentially. For example, mount 3 CF cards, then copy card 1, then card 2, then card 3. Ill look into that.


In the "color space" section, do you think in the future we could put the .LOOK file create in the Redone (or Redcine) ?

Thats another good idea. Lets see how this all plays out in the future. Ive heard of RED encrypting certain items in the latest builds, so if they encrypt this look file also it maybe difficult to incorporate the look file into this software.


How long does it take to process 4K R3D to ProRes 422 HQ on a 2.5 GHz Macbook pro in your software for exemple ?


Still working on the exact code, but you can expect similiar results as to what redline and redcine process. We are using red's official tools to do this, to maintain compatibility with R3D file structure changes and future upgrades.

[Brie]

We've been using the Data Manager during tests for a feature - I'll put updates here once we're actually shooting.
We are running on Build 15 release version.

So far, we have made a few suggestions and they have been incorporated into the program within a day or two - the response has been fantastic.
My concern is of how much use the "Project Information" files are. These are the text files that hold whatever info you have about scene, roll/reel #, cf #, etc.
These currently live with each CF cards' folder.
Personally, I believe that if this data could all be written into 1 file per day, it would make searching through it much easier if you are trying to find clips from a specific card, or scene.
Also, these files need unique names - no use to me having 100 files named Project Information.txt.

Having said all that, the program is simple to use, and is speeding up my on-set process. I haven't had any issues with the copying process yet.

Brie Walsh
Data Wrangler

[Rainer Fritz]

I installed R3Ddatamanager. That you can start mutiple copy processes saves your time that you don't have to wait till the first is ready and start the second. To generate checksums is also nice for verifying. But I don't know if I'm doin something wrong with this checksum feature, but thinking the software it self is not compairing the checksums... seems at the first look....

[Marcus Vasques Osorio]

Great program... love it

When do you think that the render feature will be available? I am especially intrested in creating QT proxies from REDline with certain color corrections. DO you think this will be available in datamanager
Marcus Vasques Osorio

[Conrad Hunziker]

My concern is of how much use the "Project Information" files are. These are the text files that hold whatever info you have about scene, roll/reel #, cf #, etc.
These currently live with each CF cards' folder.
Personally, I believe that if this data could all be written into 1 file per day, it would make searching through it much easier if you are trying to find clips from a specific card, or scene.
Also, these files need unique names - no use to me having 100 files named Project Information.txt.

The ProjectInformation.txt file is placed in the directory root for each destination. So if you just use the same destination, the ProjectInformation.txt file will be appended with each copy process. Then you would only have one ProjectInformation.txt file (one per destination at least).

Thanks for the kind remarks.

[Conrad Hunziker]

I installed R3Ddatamanager. That you can start mutiple copy processes saves your time that you don't have to wait till the first is ready and start the second. To generate checksums is also nice for verifying. But I don't know if I'm doin something wrong with this checksum feature, but thinking the software it self is not compairing the checksums... seems at the first look....

The checksums are absolutely checked at the end of each copy process at each destination, to make sure they match the source. This is, of course, as long as you have the option "check copied checksums" enabled.

You can see that during the copy process after it get to 50% done, it will say "Checking checksums in ..." for each destination.

In addition, you will find files "md5sums-*.txt", where the * is a unix time stamp, in addition to the "md5sums.txt" file. Each of those unix time stamped files contains the md5 sums from after a copy process. So if you copy a RDM directory 3 times, you will have 4 text files - the "md5sums.txt" and a "md5sums-*.txt" file for each copy process.

So as long as you have "Check copied checksums" enabled, all the above will happen on each copy process.

Of course, at any time you can check the checksum of any RDM folder, by selecting the containing folder as the source (as if you were to copy from it) and clicking on "Check R3D Checksums".

[Conrad Hunziker]

Great program... love it

When do you think that the render feature will be available? I am especially intrested in creating QT proxies from REDline with certain color corrections. DO you think this will be available in datamanager
Marcus Vasques Osorio

Thanks for the comments.

We are working on copying to/from ram for each RDM folder, which should speed up data transfers by about 30%. In addition, we are working on a way to create a queue - for those of you using cards - so you can load up many CF cards and have the program transfer them one after the other.

Then we will work with the quicktime renders. Right now we have the GUI laid out, and have completed some of the redline test renders. However, theres a lot of work to be done there still, so its a little bit off in the future. We will get it out.

Thanks.

[Conrad Hunziker]

Just an FYI, we now have a trial version available for anyone who wants to see it. The trial version is good for 30 days.

If you would like to get you hands on a copy, please use the contact form on our website: http://www.r3ddata.com/contact/

Thanks.

[ayarbro]

Photocon, is there any way to speed up the md5 checksum? at the moment, I'm trying to speed up the data moving process as much as possible. Currently thinking of using 3 xserves with caldigit's hd raid to move stuff around and speed it up.

[Conrad Hunziker]

Photocon, is there any way to speed up the md5 checksum? at the moment, I'm trying to speed up the data moving process as much as possible. Currently thinking of using 3 xserves with caldigit's hd raid to move stuff around and speed it up.

The MD5 checksums are bound by the speed of the wire (firewire, sata, etc...) and the speed of your processor. The MD5 process is simply an addition of all the ones and zeros in a file. So it has to read each bit of data to create that MD5 sum. So the faster you can make the read speed, the faster it will be.

Thanks.

[ayarbro]

Thanks for the reply! I'll get super fast machines with separate channels of ESATA then!


The MD5 checksums are bound by the speed of the wire (firewire, sata, etc...) and the speed of your processor. The MD5 process is simply an addition of all the ones and zeros in a file. So it has to read each bit of data to create that MD5 sum. So the faster you can make the read speed, the faster it will be.

Thanks.

[Trevor Meier]

Right now R3D manager calculates the MD5 checksum by reading the entire mag first, before copying. What if the software read the mag as it was copying, calculated the "read value" sum then calculated the "written value" sum on the destinations.

This would speed the copy process by 50% or more (on my system the Mag is the bottleneck - I'm writing to two independant eSATA drives and a Fibrechannel RAID)

[Conrad Hunziker]

Actually, that is already in the works. If you look on our development page, we are already working on reading/writing files to/from RAM. This would be the way to get around this issue.

The biggest issue in the entire copying process is the firewire connection to the RED Media. And if you have multiple destinations, the problem is exacerbated by every destination needing to access the same data. But by reading the files initially to RAM, you solve 2 issues:

- creating initial checksums is faster
- writing to each destination is at the DESTINATION line speed.

However, it creates a huge reliability issue. RAM is not exactly the best place to keep important files, and certainly easily exploited. In addition, you would need to have 2.5+Gb of RAM available after all the programs you are running are loaded and the base OS.

In our tests, it only increases the transfer speed by 30%. I know that 30% is a large percentage when you are dealing with time (and especially considering that multiple copies would be made during that time), but we need to program around the issues of reliability first. Because reliability is a major cornerstone of our program's success, we need to throughly work out all the possible scenarios with this before we release it.

Thanks.

[ayarbro]

You guys rock! Thanks for all the thinking photocon! I religiously use your software on every gig.

Oh...one more question...but I think you already answered it...
The CPU percentage devoted to r3d manager is fairly low, is there any way to make the r3d process a higher cpu priority? Would this speed it up at all? The reason I said you probably already answered this, is that I'm guessing the the firewire is still the bottleneck??? (like you said)

[Conrad Hunziker]

You guys rock! Thanks for all the thinking photocon! I religiously use you're software on every gig.

Oh...one more question...but I think you already answered it...
The CPU percentage devoted to r3d manager is fairly low, is there any way to make the r3d process a higher cpu priority? Would this speed it up at all? The reason I said you probably already answered this, is that I'm guessing the the firewire is still the bottleneck??? (like you said)

Thanks for the compliments.

Calculating the MD5 sums (meaning actually doing the math) is a fairly simple process, easy enough for any of todays modern processors. However, reading the numbers to add is much slower. So increasing the priority of the checksum or copy process wont increase the speed at all.

One thing to consider in addition to that, is that typically people dont just use our software on set. In addition, they may be playing back RED footage. By keeping the priority lower, more processor resources are devoted to Quicktime, and therefore you get a higher playing fps (assuming you dont have a bottleneck in reading the RED files). So it is actually beneficial to keep the priority of the R3D Data Manager right where it is at.

Thanks

[Rocco Schult]

we bought R3D Datamanager. The idea is awesome, finally somebody built something that can do that AND copy to multiple destinations.
Using it however is a different story. Having an 8core and writing to a 600MB/sec. SAN while reading from the Mags is probably among the fastest ways to work with R3D-Datamanager.
STILL, the MD5 process takes ages and in my bloody eyes way too long.
Might not be fault of R3DData/Potocon whatsoever (no pun intended, really), but we dropped/stopped using the checksum stuff.
55 shooting days, 140hrs or more, 23TB of data and 1 defect clip.
right off the drive, had nothing to do with the copy process.

Standard OS copy routine (speaking of OSX here) does the job.
Well.

[Conrad Hunziker]

You could uncheck the MD5 checkbox and still use the R3D Data Manager's ability to copy data to multiple places without any MD5 checks.

However, you need to remember that a MD5 check is not just for the first copy, but for EVERY copy. Normally you'll copy that footage again and again, over the life of the project. Backups, duplicates, SPFX, archives. By having a MD5 sum, you know everything was copied accurately - as if it were the original mag they were using.

Which leads to the next point - if you are doing any sort of bonded project, they will require some way to verify the data is valid. Having a MD5 sums fits the bill. And combined with the audit trail, it satisfies most bond company requirements.

MD5 sums may not suit your exact workflow, but you can still use R3D Data Manager. And for those that are doing bigger shows with checksum requirements, they can use it also.

[ASG]

I have two questions about your software Photocon....

When I queue up my two backup drives to offload media from my REDdrives in R3D Data Manager, it seems as though there are two simultaneous copy processes going on...

What I gathered from the earlier posts was that a sequential copy method is faster than a simultaneous one due to the hard and fast transfer speed limitations of firewire400,800, eSata etc.

Why, then, do I see two status bars indicating two copy processes in when I click START COPY?

Lastly, and totally unrelated, the software doesn't seem to be recognizing my REDdrive as such. I am not able to edit any of the project info nor check anything in "When copying from RED media:". How might I get the program to properly recognize my RED media...

note: I haven't tested the CF cards for this though. Perhaps I'll try that out tomorrow when I get a good opportunity (today's takes would have been waaaaay too long for CF)

thanks.

[Conrad Hunziker]

I have two questions about your software Photocon....

When I queue up my two backup drives to offload media from my REDdrives in R3D Data Manager, it seems as though there are two simultaneous copy processes going on...

What I gathered from the earlier posts was that a sequential copy method is faster than a simultaneous one due to the hard and fast transfer speed limitations of firewire400,800, eSata etc.

Why, then, do I see two status bars indicating two copy processes in when I click START COPY?

Im not sure where you see me say that R3D Data Manager does sequential copies, other than the upcoming queue feature. In the future version it will process sources sequentially. It always copies to destinations simultaneously.



Lastly, and totally unrelated, the software doesn't seem to be recognizing my REDdrive as such. I am not able to edit any of the project info nor check anything in "When copying from RED media:". How might I get the program to properly recognize my RED media...

note: I haven't tested the CF cards for this though. Perhaps I'll try that out tomorrow when I get a good opportunity (today's takes would have been waaaaay too long for CF)


Most likely you are not selecting the root of the drive. You need to select the root of the drive to get the program to recognize its a Red Media. For example, /Volumes/A001_0707XS/

Thanks.

[Trevor Meier]

Quick Q: I want to move a whole portable drive, containing several mags onto my edit storage.

If I've copied a bunch of mags using R3D Manager onto a portable drive, can I then select the entire volume on the portable drive and have R3D Manager copy the whole thing over for me with checksums? Or do I have to select one mag folder at a time on the portable drive?

[ASG]

Forgive my rather simplistic questions, but I want to make sure I am using your software most efficiently...

The fastest way for me to move my data around then would be to transfer to my first backup drive, run checksums, then begin a second transfer to the second backup drive, correct?

If this is the case then an automated sequential transfer function would be most welcome in a future version! Also, in addition to a % complete indicator, I'd also like to see a "time remaining" indicator as when you copy via the Finder.

thanks Photocon.

[sbaechler]

Quick Q: I want to move a whole portable drive, containing several mags onto my edit storage.

If I've copied a bunch of mags using R3D Manager onto a portable drive, can I then select the entire volume on the portable drive and have R3D Manager copy the whole thing over for me with checksums? Or do I have to select one mag folder at a time on the portable drive?

I've run into the same issue. I ended up using rsync for this task. It does make new checksums though but it is quite fast and fully automatic.
I even used the program to sync up two drives that had media scattered on each of them.
Rsync is a terminal program. No GUI. But free.

I whish there was a small utility that came with R3D manager that took care of copying media from the backup drive to other systems.

[Conrad Hunziker]

Quick Q: I want to move a whole portable drive, containing several mags onto my edit storage.

If I've copied a bunch of mags using R3D Manager onto a portable drive, can I then select the entire volume on the portable drive and have R3D Manager copy the whole thing over for me with checksums? Or do I have to select one mag folder at a time on the portable drive?

The best way, right now, is to do one mag folder at a time. I havent had the chance to explicitly test what you are trying to do.

[Conrad Hunziker]

Forgive my rather simplistic questions, but I want to make sure I am using your software most efficiently...

The fastest way for me to move my data around then would be to transfer to my first backup drive, run checksums, then begin a second transfer to the second backup drive, correct?

If this is the case then an automated sequential transfer function would be most welcome in a future version! Also, in addition to a % complete indicator, I'd also like to see a "time remaining" indicator as when you copy via the Finder.

thanks Photocon.

Fast is a relative term - because though it may be a couple of percent faster to do things sequentially, it is also slower because you actually have to be there to do it, waiting for one thing to finish and then start another. Also, there is the human factor, which is what the R3D Data Manager is trying to remove.

In a future version, we are planning on distributing r3d files via RAM, so that you get line speed (or at least system speed) on all your transfers. This should increase your transfer time by about 30% overall.

As for amount of time remaining - thats not so easy to figure out. If it was just figuring out how fast you are currently transferring and then simple division for time - that would be easy. But you have to factor in the checksum process, which heavily depends on the same processor that you are trying to calculate the time with. So while it may be better, it would also slow down the entire process.

[Conrad Hunziker]

I've run into the same issue. I ended up using rsync for this task. It does make new checksums though but it is quite fast and fully automatic.
I even used the program to sync up two drives that had media scattered on each of them.
Rsync is a terminal program. No GUI. But free.

I whish there was a small utility that came with R3D manager that took care of copying media from the backup drive to other systems.

You could even do a finder copy then use R3D Data Manager to validate the files via stored checksums. Just select the mag folder on the final destination as the source and select "check checksums".

Theres so many ways to use the R3D Data Manager, that it really does fit into just about every workflow.

[Mark L. Pederson]

Rsync is a terminal program. No GUI. But free.


Free GUI here -

http://www.opbyte.it/grsync/

[Conrad Hunziker]

Free GUI here -

http://www.opbyte.it/grsync/

Which doesnt have a mac complied binary, or even sources for the mac. So you'll have to recompile it on your system to work with mac.

[Mark L. Pederson]

Which doesnt have a mac complied binary, or even sources for the mac. So you'll have to recompile it on your system to work with mac.

Correct. It's much easier to just buy your software.

(finally got around to downloading a demo - pretty damn nice)

[Trevor Meier]

The best way, right now, is to do one mag folder at a time. I havent had the chance to explicitly test what you are trying to do.

Copying a whole drive worth of mag folders would be really useful for us... our workflows all have shows sending 2-3 "film breaks" a day back to the lab to have them backed up and processed. The mags stay on the camera truck, and duplicates are sent on portable drives. Maintaining the audit trail in this flow with a full drive copy would be very handy.

[Jeremy Newmark]

You could even do a finder copy then use R3D Data Manager to validate the files via stored checksums. Just select the mag folder on the final destination as the source and select "check checksums".

Theres so many ways to use the R3D Data Manager, that it really does fit into just about every workflow.

Hey Conrad,

Quick question, when you do this it just runs through and creates another text file for that date without telling you that it's finished. If there was a problem and they did not match, does it alert you about this or do you have to physically go in and compare the text files?

[Nir Shelter]

R3D data manager proved to be a very good tool tested it on set today. It was robust and easy to use. Also compliments to Conrad very quick replies to my queries with easy to understand trouble shooting.
Looking forward to the QT import window very much!

I had a problem entering info into the production info menus. Conrad pointed out that this can be done only when the source directory is a red drive or CF. Tried it again directly from the RED media and it worked. Wondering if in future this will be enabled for copied data as well as directly on a red media?
I can see many applications of this in post situations where a drive is received with copy/pasted data on it and we need to start a checksum process with out the original media.

[Conrad Hunziker]

Hey Conrad,

Quick question, when you do this it just runs through and creates another text file for that date without telling you that it's finished. If there was a problem and they did not match, does it alert you about this or do you have to physically go in and compare the text files?

We will update this in a future version, but right now it will only alert you if there is an error.

[Jeremy Newmark]

We will update this in a future version, but right now it will only alert you if there is an error.

Great! Thanks, it's good to know that it will alert you now, if there is a problem. We've been testing out the software on a 2 camera shoot for the last 2 weeks and it works great. We'll be purchasing it soon. Thanks again.

[Conrad Hunziker]

I had a problem entering info into the production info menus. Conrad pointed out that this can be done only when the source directory is a red drive or CF. Tried it again directly from the RED media and it worked. Wondering if in future this will be enabled for copied data as well as directly on a red media?
I can see many applications of this in post situations where a drive is received with copy/pasted data on it and we need to start a checksum process with out the original media.

R3D Data Manager was designed to be an end to end solution. As such, the only place that it makes sense to enter that production info is on set. Afterwards you would need to view each clip individually to get all the info there, especially the script scene numbers. In addition, in post, this program may not even be used for the same production at the same time, let alone days or scene numbers.

And in addition, if you are creating checksums from copied data - the you are only able to verify the data to that point. Who knows if the first copy was correct, or even complete?

[Steve Sherrick]

I am using R3D Data Manager on set right now, working like a charm. It actually has flagged some files that had potential problems (turns out they were okay) and I feel a lot more confident on set using a tool like this. When I hand a drive back to the camera crew, the files have been backed up to three destinations, checksums have been verified, and everyone is confident that the media is safe and we can move on.

I just used it on a movie last week as well, and knowing that the talent was probably not going to be able to come back to do re-shoots (they are in their 90s), again I felt like we had a good backup plan in place. So no worries.

And, not to plug the product anymore, but Conrad fielded emails from me about a problem I was having when I was testing the program and it turns out it was not the software but the way the Red Drive was connected through a Wiebtech Combodock. Once I removed that variable, things were fine, but he was quick to respond to me about the issue to help me troubleshoot.

[Scott Mason]

Conrad-

Where I see this useful is for when we get back to the office. We use the software on set while we are downloading in the field. It does the checksum and we know we copied off the red media to our hardrives in the field. Now when we get back to the office we take all the media from the hard drives and copy it to the server, then it would be great to use the software again to make sure it is copied correctly. This would be done from a non red drive to a non red drive and still needs to be verified.

[Conrad Hunziker]

Conrad-

Where I see this useful is for when we get back to the office. We use the software on set while we are downloading in the field. It does the checksum and we know we copied off the red media to our hardrives in the field. Now when we get back to the office we take all the media from the hard drives and copy it to the server, then it would be great to use the software again to make sure it is copied correctly. This would be done from a non red drive to a non red drive and still needs to be verified.

I dont see this as an issue. Just use the program again to do the copying. It is designed to work like this. Files are always verified as long as you have the option enabled.

[Scott Mason]

Sorry I misunderstood something from your post, when you said where I thought it would only verify media from a red device. That is great it will work that way too.

[Nir Shelter]

And R3D Data Manager is a great end to end solution.
What I am saying is it would be great as a solution for situations in which a DIT is not on set (stupid I KNOW!!! but some producers don't know ow any better).
I was on a low budget feature last year and they could afford me for only half the shoot days. The other half I had to teach someone ells how to copy files they delivered the HDD to me at the end of every days shoot for back up and redundancy. A tool like this with the production info available at any post stage would be brilliant.

[Sean Sweeney]

Thanks to Conrad and everyone who had a hand in r3d data manager - we've tested it out on a couple of shoots now, and even on a lowly macbook it performs very well!

The peace of mind is worth the price alone. I'm excited for the new features.

[Craig Meadows]

I'm sold - just ordered - can't wait - too cool!

[William Wedig]

Can Data Manager do a sorting to the destination? Like have a destination for 4k 2:1 and then a different for 4k 16:9, then a 3rd for 3k, 2k, etc etc?

This would help out a lot when making my proxies for offline edit.

[Conrad Hunziker]

Can Data Manager do a sorting to the destination? Like have a destination for 4k 2:1 and then a different for 4k 16:9, then a 3rd for 3k, 2k, etc etc?

This would help out a lot when making my proxies for offline edit.

Right now R3D Data Manager does not sort any footage. Im curious to know exactly your workflow so I can see how this functionality would help.

Thanks.

[William Wedig]

My projects are shot with anamorphic lenses btw, so the unsqueeze adds more steps than the average project. Although this could theoretically help out other people as well.

The workflow goes like this:

1. Backup footage accordingly from camera

2. Use RedRushes (or Data Manager when it's implemented) to export offline clips for edit at a decent resolution with a crop and an unsqueeze (cropping a 16:9 or 2:1 frame into a 1.20 Anamorphic extraction then unsqueezing it to a 2.4). ProRes or DVCProHD.

3. Take the ProRes or DVCProHD files into Quicktime and down convert them into a normal frame size (1280 X 720 or 960 X 720) with black bars at the top and bottom squeezing it into the frame using QT's Preserve Aspect Ratio->Letterbox to inset the 2.4 frame into a standard 16:9 frame.

4. Edit in FCP with offline DVCProHD or ProRes files.

5. Conform with Crimson, color correct with Color, finish.

--

There are some problems with that workflow that could be smoothed out (for instance saving duplicates of the footage between RedRushes and Quicktime) and it'd be nice to only have to run it through one program instead of two. Also how good is the down conversion in QT? From what I saw it looked totally fine for offline.

The reason the black bars and standard frame are so important to me is that it will speed up editing months down the road (my projects are all features meaning at least 4-6 months of editing) and will allow for minimal FX and color correction along the way in the offline, to know that the choices you are making as to which footage to use for what will properly work later.

By insetting the frame into a standard frame will free up a good portion of the CPU and keep it from having to render every clip in the timeline on the fly. This is the part of my work flow that I think most people can appreciate because it will really make editing less painful for editors.

PS: Each different resolution requires different math through Redrushes which is where the sorting of files upon import makes sense. It could be as simple as having a check box called "Sort according to resolution/frame size" and then in the output folders it just creates a folder for each different type (4k 2:1, 4k 16:9, 3k 2:1, etc) and that way when we go to down convert you can just select a whole folder and be sure that all the clips in there are the correct setting for the output.

[SF Geek]

I can't seem to input any shot info, project info, or red media info. My preferences option is greyed out in the dropdown menu.

[Conrad Hunziker]

I can't seem to input any shot info, project info, or red media info. My preferences option is greyed out in the dropdown menu.

You need to select the root directory of a valid RED Media - Red Drive or CF card.

[Dave French]

Has anyone used ShotPut?

http://www.imagineproducts.com/ShotPut_RedEdition.html


It looks like a similar program to R3D Data Manager. I am just trying to decide right now which one to get and was wondering if anyone has ever compared the two.

Thanks
Dave

[Conrad Hunziker]

One of the biggest differences I see is that it doesnt do checksum verification - only byte based comparisons. So once the red media is erased, you will have no way of ever re-verifying the r3d file again. Basically, if you plan on archiving at all, you should have a checksum to verify that the data is correct. It is also a requirement of most bonded shows that a checksum - from the source media - be generated.

edit: I forgot to add that we have a couple of nifty features on the horizon. Automated renders and a queue system are around the corner - currently in beta testing.

[Rainer Fritz]

Hi Conrad !

One question: If I do the checksum verificationI see the process bar going to 100 percent and if it is finished I get no response message. Would it be please possible to get a popup window like when all the media is copied?

thx
rainer

[Bruce Ingram]

Question for current users:

We are having issues with our MacBook Pro Kernel Panicking on us during the transfer of files in R3D Data Manager and I've narrowed the issue down to something in the chain of Sata devices. So, I was hoping to see if anyone who has been successfully using the software has used the Tempo SATA ExpressCard/34 from Sonnet without any issues and/or what SATA expresscards folks have used with success in conjunction with R3D Data Manager.

Any info would be great!

[Steve Sherrick]

I just picked up a Sonnet Tempo card recently but have not tested yet with R3D Data Manager. I will be doing that soon though.

[Conrad Hunziker]

We have had great success with the Rosewill eSATA express card. Theres another one that I cant remember off the top of my head, but its driver wouldnt let you unmount volumes from within finder.

[Conrad Hunziker]

Hi Conrad !

One question: If I do the checksum verificationI see the process bar going to 100 percent and if it is finished I get no response message. Would it be please possible to get a popup window like when all the media is copied?

thx
rainer

Ive mentioned before that this will be in a future version. It does alert you now if there is an error.

[Rainer Fritz]

Conrad, please one more question....

when I do the checksum creation, then copy, and after that the checksum check, it does it with the camera disk again. The backup disks are in idle and doin nothing... should it not check the checksums from the cam drive with the backup drives???? confusing a little bit....

thx
rainer

[Joel Kaye]

Is there a way to copy select files and folders only?

For instance, I had a checksum error on 2 files. Seems like I should be able to go back and copy just those two files or folders but the app seems to limit me to the root R3D directory.

I also would like to see the destination paths be editable. I change the destination folder for every reel so I can give some real world names to the footage. So the whole preferences thing doesn't save me any time as I have to remove the path and add a new one by browsing to the new folder every time. Kinda annoying.

Also, when you get an error there should be a one click method to either recopy those files or take you to those files and folders so you can inspect them quickly to see if they are corrupt.

The whole checksum creation button is unclear to me. Do I have to that before I start the copy every time? I think that button should be eliminated and happen automatically when the Start Copy button is pushed if the checksum option is chosen.

[Paul Leeming]

Question for current users:

We are having issues with our MacBook Pro Kernel Panicking on us during the transfer of files in R3D Data Manager and I've narrowed the issue down to something in the chain of Sata devices. So, I was hoping to see if anyone who has been successfully using the software has used the Tempo SATA ExpressCard/34 from Sonnet without any issues and/or what SATA expresscards folks have used with success in conjunction with R3D Data Manager.

Any info would be great!
I had kernel panics with that card using Leopard and sold it as a result. Had nothing to do with R3D Manager as I wasn't using it. Most eSATA ExpressCards have the same chipset and I tried another one (Rex RS30 or something) with same chipset with same results.

Haven't used eSATA cards since then and went back to FW800.

If anyone can recommend a card with a different chipset in it to those two it'd be appreciated....

HTH

Paul

[Cameron Preyde]

Conrad, please one more question....

when I do the checksum creation, then copy, and after that the checksum check, it does it with the camera disk again. The backup disks are in idle and doin nothing... should it not check the checksums from the cam drive with the backup drives???? confusing a little bit....

thx
rainer

I'm pretty sure it does that while you're copying. You don't need to check the checksums again after the copy.

[Conrad Hunziker]

Conrad, please one more question....

when I do the checksum creation, then copy, and after that the checksum check, it does it with the camera disk again. The backup disks are in idle and doin nothing... should it not check the checksums from the cam drive with the backup drives???? confusing a little bit....

thx
rainer

I dont get why you are doing a checksum check again. As long as you have the option enabled, it will do it on each destination immediately after the copy has finished - namely from 50 to 100% done on each destination. In fact, once each destination says "checking checksums", you could remove/unmount the source red drive.

[Conrad Hunziker]

Is there a way to copy select files and folders only?

For instance, I had a checksum error on 2 files. Seems like I should be able to go back and copy just those two files or folders but the app seems to limit me to the root R3D directory.

You are thinking of it backwards. Instead, just delete the offending destinations that had copy/checksum errors. select "ignore all" in the "if RDC folder exists" option, and hit copy again. It will find the deleted folders and only copy those. Since you deleted the ones that had errors, it will effectively just re-copy those.


The whole checksum creation button is unclear to me. Do I have to that before I start the copy every time? I think that button should be eliminated and happen automatically when the Start Copy button is pushed if the checksum option is chosen.

If the checksums are not being automatically done when the source is selected (i.e. BEFORE you even press the "start copy" button), then you are doing something wrong. The source checksums (ie. on the Red Drive or CF card) are done automatically when R3D Data Manager detects you selected a valid Red Drive. So while you are entering other info, its already creating the checksums.

[Harry Clark]

I use a Sonnet Dual eSata Expresscard adapter and it works fine. Certainly no kernel panics!
Conrad,
Terrific program! An "estimated time remaining" countdown would be a welcome addition.
Cheers,
Harry

[Joel Kaye]

You are thinking of it backwards. Instead, just delete the offending destinations that had copy/checksum errors. select "ignore all" in the "if RDC folder exists" option, and hit copy again.



If the checksums are not being automatically done when the source is selected (i.e. BEFORE you even press the "start copy" button), then you are doing something wrong.

Ok, got the re-copy part. Thanks. But considering I've got a zillion years of computer experience and my first reaction was just to go select the 2 bad files...

The best solution would be to have the program flag files that were not copied properly and offer to show the destination files in the finder and offer a one click "copy these files again" option. I really don't want to have to do any thinking if I have a bad copy.

If checksums are automatic why is a create R3D checksums button right next to the select source dialogue? Sounds like it would be rarely used. If so, consider moving it to a menu item and call it recreate R3D Checksums. Right now it feels (to me anyway) like it is part of the process due the prominent placement of the button.

[Bruce Ingram]

Hey, just an update to my earlier post about issues with the Sonnet Tempo ExpressCard. Following Photocon's post, we returned that crad and ordered the Rosewill eSATA ExpressCard (model RC605) and have tested it without any issues.

While I haven't used it on set yet, I tried multiple scenarios of how I may need to copy while on set with Mags loaded with lots of shots, a few shots; doing checksums, not doing them; verifying them and not all without any hiccups at all.

With our hardware (MacBookPro, 7200 RPM drive, Lexar FW800 CF Reader, Rosewill eSATA ExpressCard, dual 500GB G-Tech G-Drive Qs) my numbers are currently showing an average 1:2 copy rate on Mags that I do checksums on (4 mins of footage takes about 8 mins to copy) and a close 1:1 copy rate without checksums, although this has a variation of anywhere between 10 to 30 seconds.

I can't say this is 100% rock solid, but it is consistant with what I've been experiencing so far and well within a workable time. I also can't guarantee that the Rosewill will work for everyone, but it is working for us. Like Paul (visceralpsyche) our laptop had Leopard on it and I wouldn't be surprised if that was part of the issue.

[Nick Shaw]

It would be very useful in a future version to be able create and verify checksums after copying, when checksum was not switched on during copy. Often under the pressure of a shoot the time penalty for copying with checksums may be too much. I'd like to be able to point the app at a Red Drive and a copy folder at the end of the day and have it create and compare checksums. Obviously this only works if you haven't erased the Red Drive! I personally prefer not to erase drives until the end of the shoot, and do regular incremental backups.

[Conrad Hunziker]

It would be very useful in a future version to be able create and verify checksums after copying, when checksum was not switched on during copy. Often under the pressure of a shoot the time penalty for copying with checksums may be too much. I'd like to be able to point the app at a Red Drive and a copy folder at the end of the day and have it create and compare checksums. Obviously this only works if you haven't erased the Red Drive! I personally prefer not to erase drives until the end of the shoot, and do regular incremental backups.

You can do that, in a way, right now.

You can disable checksums and then create them later using the "create R3D checksums" button.

You can disable checksum checking after copy, then select a destination RDM folder as the source and check the destination checksums then.

The only thing you cant do is have R3D Data Manager copy the md5sums.txt file to the RDC folders. It only does that during a copy process. So you would need to copy those to the destination directories manually.

In the future actually, there will be no penalty for the source checksum. That will be done automatically while the file is read.

[Trevor Meier]

Conrad, any ETA on a new version with some of the requested speed improvements?

[Conrad Hunziker]

No ETAs on the speed improvements. The speed is really determined by how fast you can read/write, not the program per-se.

The quicktime render and queue system are in beta right now.

[Nick Shaw]

You can do that, in a way, right now.

You can disable checksums and then create them later using the "create R3D checksums" button.

You can disable checksum checking after copy, then select a destination RDM folder as the source and check the destination checksums then.

The only thing you cant do is have R3D Data Manager copy the md5sums.txt file to the RDC folders. It only does that during a copy process. So you would need to copy those to the destination directories manually.

That's what I have been doing, but if I don't get a chance to create checksums until the end of the day there can be a lot of files to copy manually to different folders. I need to get more up to speed on writing shell scripts to do that kind of thing!


In the future actually, there will be no penalty for the source checksum. That will be done automatically while the file is read.

That will be extremely useful - always creating checksums at the earliest opportunity, but not verifying them until time allows, without the time taken to read the source file twice. I look forward to the update.

[Trevor Meier]

The speed is really determined by how fast you can read/write, not the program per-se.

Creating the checksums during the read/transfer would increase the speed significantly - by about a third. is this still a ways off?

[Conrad Hunziker]

Creating the checksums during the read/transfer would increase the speed significantly - by about a third. is this still a ways off?

Semantics perhaps, but it does not increase the speed one iota. It just decreases the time by doing two things at once.

And it is a ways off still mostly because RAM is not a friendly place for data to be.

[Nick Shaw]

Semantics perhaps, but it does not increase the speed one iota. It just decreases the time by doing two things at once.

Surely it doesn't just do two things at one, it does one very time consuming thing (ie reading the source file) once instead of twice. Or am I misunderstanding?

[Conrad Hunziker]

Surely it doesn't just do two things at one, it does one very time consuming thing (ie reading the source file) once instead of twice. Or am I misunderstanding?

Im not going to divulge the exact innards of how it works, especially since it is not released yet. However, suffice to say, it IS (or will be?) doing two things at once.

[Damon Meledones]

Wouldn't creating the checksums while copying remove the ability to detect random, silent, read errors? By doing a checksum and then copying the data, you are not only verifying that it was written correctly, but also much more certain that it was also read correctly. Granted, we always scrub through all the clips, and random, silent, read errors are few and far between, it's still something to think about.

[Nir Shelter]

Sorry to post off topic hear but I have a question. I want to be able to copy to 2 separate destination directories simultaneously. But when I set R3D DMNG to do so I get checksum errors at the end of the process on almost all the clips. My setup is:
2 X HDD -> express34 to 2XFW800 -> Macbookpro <- Red drive to FW800

Copying to the 2 HDD seams to always result in a checksum error at the end of the copying process. This also happens when the drives are daisy chained to each other.

Is there a way to copy to 2 separate drives at the same time?
Thanks.

[Conrad Hunziker]

R3D Data Manager usually exposes some sort of other unreliable system in your chain. Something is unreliable, and its causing checksum errors. I would advise to try different things to try and isolate the issue. Most issues happen with incorrectly made expresscards and/or drivers.

To be clear, R3D Data Manager is expressly designed to copy to 1, 2, 3 or 4 drives at the same time. Copying to 2 drives is *NOT* the issue.

[Nir Shelter]

Hi Conrad I know that copying to 2 drives is not the issue and there for am asking whether you can help me make it work on my system.
Copying to one of the 2 connected via the express card drives at a time works and the express card adapter seams to work fine with other software as well. Copy to 2 drives daisy chained to each other and connected via the native FW800 port doesn't work. Nor does copying to 2 drives when connected to the express card daisy chained or separately to the 2 FW800 ports on the adapter.
The Macbookpro has 2GB RAM, 2.33GHZ core 2 duo and OS X 10.4.11, any helpful suggestions would be great.
Thanks.

[Conrad Hunziker]

Just because your drives work with other software doesnt mean its up for the task of handling red footage. Obviously there is something in the chain that is not reading or writing the footage correctly. The ones and zeros are getting misplaced. That alone, for me, would be reason to suspect every part of the chain.

You still need to investigate the drives themselves, the cables you are using, even the computer you are using. Any one of them could be responsible for this. Have you checked the logs?

[Nir Shelter]

The drives are bran new Taurus raid II set to raid 1, the cables are new FW800 and the computer has worked flawlessly to date. I haven't read through the logs so will try a copy to 2 drives again just to get a log and read it. Will also try it with un raided drives may well have an effect.

[Conrad Hunziker]

When you find the log, please use the "send feedback" option in the R3DManager menu and copy/paste the relevant portion of the log there. By doing this, we will get other valuable info about your system.

[Nir Shelter]

I would like to praise the latest release of R3D Data manger V.3. I really like the new lay out and the queue option for source directories is great. Very happy with the new version thanks Conrad.

P.S
I will send the next log I get from a shoot with Red drives and 2 FW800 HDD as it seams as if the problem I was referring to was due to a combination of a dual FW800 to express 34 adapter and R3D DMNG.

[Conrad Hunziker]

Just wanted to let people subscribed to this thread know about the new 3.1 version:

http://www.reduser.net/forum/showpost.php?p=300373&postcount=11

[Conrad Hunziker]

Just an update to those who are watching this thread. Version 4.0 announced:

http://www.reduser.net/forum/showpost.php?p=305665&postcount=1

[Steve Sherrick]

Looks good!

Will try it out soon.

[zak forrest]

are there any other programs out there that can assure me im making good copies from a red drive to my RAID or is r3ddatamanager the only one?

How long does it normally take to make copies from a nearly full red drive?

its been on zero percent for what seems like an eternity.

[Conrad Hunziker]

Well - a nearly full hard drive indicates bad data management, to be honest.

To make an accurate copy you need to read the file 3 times. The first reads the file to make an initial checksum, the second does the copy, the third does the verification (reading from the destination). Because of this you can expect a full Red Drive to take a day, because firewire 800 is only so fast. Hopefully you have setup your system for speed, and only are using the firewire 800 to the Red Drive, and are using SATA to the other drives.

As all your eggs right now are in one basket with all your footage on one drive. If a hard drive fails, you are out of luck. Im mentioning this because you really need to ensure that you are downloading more often. When you do, it wont take as long either.

[zak forrest]

Thanks for the quick response dude.

It was impossible to download the data that often on this job. A three camera shoot of an all day music festival, each camera had two red drives, every two and a half hours we would each switch to the next drive. There were five minute (at the most) breaks between each act. There was no other option with our budget (which was zero)

[Conrad Hunziker]

Believe me, I understand budget restraints. However with just one more drive per camera (two to be safe), you could implement a very decent data backup strategy.

[zak forrest]

can you explain what that backup strategy would be ?

[zak forrest]

well, can you? i'd love to hear your thoughts on that backup strategy


Anyway, r3d data manager has been saving my ass the past few days.

so far only one request:

when i view the copy results, it would be great if i could resize that window with the log in it so that each line of the log fills up one line and goes across my screen. its kind of annoying to figure out where each line ends and the next begins when im scrolling through the log.. its copying now so i cant test it, but i think when you resize the log window it just gets bigger but the square window of text that displays the log stays the same size! can you fix that?

[Conrad Hunziker]

well, can you? i'd love to hear your thoughts on that backup strategy

Honestly, I make a living figuring out data workflows and backup strategies for films, so Im not going to give away all the secrets. However, given that you need to change batteries about every 90 minutes, and music act rarely do sets longer than 40 minutes, its not hard to see where 4 drives would allow you to do a usable workflow where your data is protected and backed up and at the end of the day you have less than 150Gb on your last drive to download.

Since writing R3D Data Manager, Ive heard lots of stories about how they werent able to backup because of X, Y, and Z. The most often heard is that they had a 2.5 hour sitdown interview with a VIP - no time to change drives. How would they have done that same interview on a Panasonic or Sony offering? My opinion is that they overlook opportunities for downloading because there is still space on the drive - but then get mad when it takes a while to download. Whereas, if they had taken those opportunities to download during the shoot, they would have not only saved their valuable data from being lost due to drive failure BUT ALSO spread out the amount of time it takes to move the data over the course of the shoot.

I really believe that if you dont figure in a data backup strategy into your shoot, and make it a priority, dont bother pushing the red button. As the data sizes increase (6k, 9k, Epic & the lot) this will become MUCH more important.


when i view the copy results, it would be great if i could resize that window with the log in it so that each line of the log fills up one line and goes across my screen.

Already fixed in the next release.

[Conrad Hunziker]

Real world proof why I think its so important to backup as often as possible:

http://www.reduser.net/forum/showpost.php?p=339331&postcount=7

[Troy Smith]

Hi Photocon,

Is the windows version still on for this month?

Thanks
Stricko

[Conrad Hunziker]

Honestly, its one of those things - we keep adding features to the Mac side, mostly from user input, and realize those features change the way things on the windows side would be done significantly enough to just do it now rather than do the effort twice.

The upshot is that the mac and windows versions will be very close from the start. The down side is that it will be next month.

Thanks.

[Troy Smith]

that's cool, I need it for a shoot coming up in about 4months, so it will be ready for me, cool.

[Dave Larson]

I have the trial version, on a Mac Book pro
when I select the RDM Source Folder on the Red Drive, It shows up in the queue, but, below under Red Media, Shot Info, and Project Info it says "Not Valid Red Media"
Am I doing something wrong? or is this a function of the trial version?
The RDM Folder is the only Folder I can add to the queue,No other Sub Folders(RDC)? is this correct?
BTW Great program! I just want to make sure I'm using it correctly

[Conrad Hunziker]

You should select the root of the red drive, not the RDM or RDC folders, to get it to validate it as a red drive. You can select a RDM or RDC folder and just transfer those folders.

[MichaelP]

Conrad -

Do you think you can add the full set of DNxHD formats to the renders since REDline now offers that?

Thanks!

Michael

[Jason Diamond]

that would be great! thanks for suggesting that Michael.

[Conrad Hunziker]

Do you think you can add the full set of DNxHD formats to the renders since REDline now offers that?

The code is already in the next release. We are finishing up some testing and should have it out after the holidays.

[jahno]

hey there,
we bought the software only 3 weeks ago and tested it one week. great tool!
no problems within one week of shooting unless one:

Several Clips i copied resulted in copy & checksum error!
i was searching for the failures in the protocol... NOTHING to find. tried a few times again with the sam clips. a few worked out well a few did not.
still no error to find in the protocol only the queue window and the signalthing told me some red errors...

i copied the clip manually at last...

that was the on set bug i figured out. we the same problem back in the office but i am not sure if it concerns the same clips... (clips we're playalbe and fine to work with in RC and redrushes)

Our system:
latest macbook pro 17" 2.5ghz, 4gb ram, ....
wd 2tb raid1
lexar firewire cf cardreader
another backup hdd
and finally in the office a san.

any ideas why this happened?

best regards
jahno

[MichaelP]

The code is already in the next release. We are finishing up some testing and should have it out after the holidays.

Excellent, thanks! Will it also export an ALE file in addition to currently supported file exports?

Michael

[Conrad Hunziker]

hey there,
we bought the software only 3 weeks ago and tested it one week. great tool!
no problems within one week of shooting unless one:

Several Clips i copied resulted in copy & checksum error!
i was searching for the failures in the protocol... NOTHING to find. tried a few times again with the sam clips. a few worked out well a few did not.
still no error to find in the protocol only the queue window and the signalthing told me some red errors...

i copied the clip manually at last...

that was the on set bug i figured out. we the same problem back in the office but i am not sure if it concerns the same clips... (clips we're playalbe and fine to work with in RC and redrushes)

Our system:
latest macbook pro 17" 2.5ghz, 4gb ram, ....
wd 2tb raid1
lexar firewire cf cardreader
another backup hdd
and finally in the office a san.

any ideas why this happened?

best regards
jahno

If you get errors on the same clips over and over, and the clips play back fine, it may be a read error on the files when the checksums were first generated. You can look in the md5sums.txt file and compare those to the md5sums-****.txt files and see the difference. If all the md5sums-***.txt files have the same sum for the same file, but its different in the md5sums.txt file, then it was a read error on the initial checksum generation. You can use the "create new checksums" button to solve that issue.

This goes to show that errors can happen at any time - 1st read, 2nd read, transfer, 1st write, etc... By reading multiple times you can be sure that the data is correct.

[jahno]

If you get errors on the same clips over and over, and the clips play back fine, it may be a read error on the files when the checksums were first generated. You can look in the md5sums.txt file and compare those to the md5sums-****.txt files and see the difference. If all the md5sums-***.txt files have the same sum for the same file, but its different in the md5sums.txt file, then it was a read error on the initial checksum generation. You can use the "create new checksums" button to solve that issue.

This goes to show that errors can happen at any time - 1st read, 2nd read, transfer, 1st write, etc... By reading multiple times you can be sure that the data is correct.

hey! thanks for your answer. i compared the errors in the last few weeks of shooting...
the copy protocoll said "reference checksum not available" or something similar.

here's what i did:
i copied the files from CF cards to a RAID1 external HDD. then back in the office i copied to our SAN. of course both times via the manager...
i was wondering why only about 40 to 50% of the reference checksums were missing. my first idea was that the reference is on the CF card... but must be impossible because not every reference was missing.

any ideas?
thanks
jahno

[Conrad Hunziker]

hey! thanks for your answer. i compared the errors in the last few weeks of shooting...
the copy protocoll said "reference checksum not available" or something similar.

here's what i did:
i copied the files from CF cards to a RAID1 external HDD. then back in the office i copied to our SAN. of course both times via the manager...
i was wondering why only about 40 to 50% of the reference checksums were missing. my first idea was that the reference is on the CF card... but must be impossible because not every reference was missing.


Theres something you are doing that is not standard. Mounting the CF cards as read only or the reader is having an error. I would try different CF reader hardware or make sure you are allowing everything to complete properly.

[Conrad Hunziker]

Just an FYI to those watching this thread, we just released version 4.2:

http://www.reduser.net/forum/showpost.php?p=362168&postcount=1

[jahno]

Theres something you are doing that is not standard. Mounting the CF cards as read only or the reader is having an error. I would try different CF reader hardware or make sure you are allowing everything to complete properly.

Okay... hmm. what is not standard in our process?
We're using the lexar fw800 cf reader. i think that should be the best to get hm?
so i should mount it in read only? or i should not? i don't even know how to mount them in read only.

i am not sure if i explained the problem proper...
there's is no problem when we copy from CF to HDD_1. the problems appear when the CFs are already sleeping in their little peli case. the problems appear when we do a back up from hdd_1 to hdd_2

[Conrad Hunziker]

Then the issue may be with your first hard drive. Please send us an email with the log of the transfer and a screenshot of the folder contents for a RDC folder that does not copy correctly.

There still is something you are not doing correct, which we need to figure out.

[Nir Shelter]

Just an FYI to those watching this thread, we just released version 4.2:

http://www.reduser.net/forum/showpost.php?p=362168&postcount=1

Thanks Conrad :)

[Nir Shelter]

I installed the V4.2 update but was unable to export from the QT render window CSV files of the metadata. I did the same thing as I have been doing with V4.1 selected the source R3D files and then clicked the CSv tab but there was no response tried it several times as well as relaunching R3D Data Manager tried the TXT and PDF options but the same thing happened (or didn't happen...).
I ended up reverting to the older version I had installed to continue working, is this a bug or should I be doing something different after the update in order to achieve this?

[Conrad Hunziker]

I installed the V4.2 update but was unable to export from the QT render window CSV files of the metadata. I did the same thing as I have been doing with V4.1 selected the source R3D files and then clicked the CSv tab but there was no response tried it several times as well as relaunching R3D Data Manager tried the TXT and PDF options but the same thing happened (or didn't happen...).
I ended up reverting to the older version I had installed to continue working, is this a bug or should I be doing something different after the update in order to achieve this?

Please see this thread:

http://www.reduser.net/forum/showthread.php?t=25171

[Nir Shelter]

Thanks Conrad much better now, thank you for an awesome app!!!
Like the updates will post any suggestions and/or comments if they come up.

[Ivan Cortazar]

Hi Conrad,

I bought R3D manager a few weeks ago and I think it is a great application. I don't know if it has been discussed earlier or if it makes any sense, but it would be great if R3D manager could do more than R3d files.

I say this because sometimes, after backing up from Red Card or Drive to external drives, I work on one of my drives. And just for organizing issues I end up placing on some folders some rendered files, jpegs that I used for compositing, and It would be great if R3D manager could copy to another 3rd hard drive (when the project is done) the whole drive, including R3d files, rendered files, or jpegs.

I guess this would be another application, like synk or something like that. But it would be great that R3d manager could do checksums on everything, not only the 3rd manager. I apologize if this doesn't make any sense.

Thanks,

Ivan

[Conrad Hunziker]

Hi Conrad,

I bought R3D manager a few weeks ago and I think it is a great application. I don't know if it has been discussed earlier or if it makes any sense, but it would be great if R3D manager could do more than R3d files.

I say this because sometimes, after backing up from Red Card or Drive to external drives, I work on one of my drives. And just for organizing issues I end up placing on some folders some rendered files, jpegs that I used for compositing, and It would be great if R3D manager could copy to another 3rd hard drive (when the project is done) the whole drive, including R3d files, rendered files, or jpegs.

I guess this would be another application, like synk or something like that. But it would be great that R3d manager could do checksums on everything, not only the 3rd manager. I apologize if this doesn't make any sense.

Thanks,

Ivan


Right now R3D Data Manager will copy any file placed into RDC or RDM folders. Copying everything above that is untested, and it wont generate checksums on that material.

You could just do a finder copy and then check the checksums of R3D files. Just set the destination of the copy as the source and click on the "check checksums" button. Then all your data will be copied AND R3D files will be checksum checked.

[Rob van Gelder]

Having played a bit with R3D manager and also Clipfinder, i found that there is a big difference in rendering quicktimes:

With the same settings I find that clipfinder renders quicktimes at about max. 16 fps on my MBP, the quicktime render window from R3D manager goes to appox. 4-5 fps.

What could be the reason of this?

Rob

[Conrad Hunziker]

Having played a bit with R3D manager and also Clipfinder, i found that there is a big difference in rendering quicktimes:

With the same settings I find that clipfinder renders quicktimes at about max. 16 fps on my MBP, the quicktime render window from R3D manager goes to appox. 4-5 fps.

What could be the reason of this?

Rob

Clearly, your settings are not matching up. The biggest reason may be the render resolution. In our testing R3D Data Manager is about 5% faster than RedRushes, so its hard to believe that both would be slower than clipfinder.

[Kristin Stewart]

Hi Conrad,

Any news about the release date of the QT batch conversion option for the windows version ?

Thanks,

Kristin

[Conrad Hunziker]

Kristin-

Still having issues with the copying portion of R3D for windows - the batch conversion is a ways off still, it wont be in the first windows version.