Thread: "Freeish" 0K-2K-4K-6K+ Workflow for Download now...

Reply to Thread
Results 1 to 3 of 3
  1. #1 "Freeish" 0K-2K-4K-6K+ Workflow for Download now... 
    Senior Member Dan Hudgins's Avatar
    Join Date
    Feb 2007
    Location
    San Francisco, CA USA
    Posts
    1,370
    Thanks to everyone here on REDUSER for their help over the last year and a half or so with sample files and answers.

    >>Note, Some tasks or programs used require Windows ME (tm) or Windows XP Home SP3 (tm) with the FreeDOS FAT32 (tm) disk mounted as a slave drive. Things may work better if all disks are formatted FAT32, which should work up to 2TB per disk.<<

    I have continued development of my "freeish" uncompressed DI system, keeping in mind RED ONE (tm) users who might want to use it for Color Correction, sound mixing, editing, or all of the above and more...

    I think I have the 16 bit RGB TIF file import and export working with files of up to 6K or more, although I would leave it up to you as to how you are going to store 126,720 times 120MB frames.

    I have made changes to DANCAD87.EXE (tm) for running it under OS that shpport 2TB disks, these seem to be Windows XP Home SP3 (tm), Windows ME (tm), and FreeDOS FAT32 (tm).

    There seem to be some issues with XP (tm) and SVGA graphics modes on some but not all computers, but since frame processing can be done in just text mode without the graphics above 640x480 it seems a good choice for large disk arrays.

    I am looking into Windows ME (tm) since it may work better for tasks that require graphics and network support, and sync playback in editing.

    I have made some adjustments to the programs for using with FreeDOS FAT32 (tm) with large disk support. I have been testing FreeDOS FAT32 (tm) it seems that I may work out for use in Grading, and some sync playback for editing, as well as for use in frame processing (color correction and resize, letter box, etc.) in a render farm of junk computers upgraded with new low cost motherboards, CPU, and memory. Some of the junk computers I have for my render farm do not seem to work with 500GB EIDE disks, so I will try new mother boards when I can get some. On newer computers it seems to work. Since most of the time is used in harddisk data read and write the CPU speed is only part of the time, so anything between 1GHz and 2GHz should be usable, if you want to spend more you might get some speed, but it may be better to have 10 2GHz computers than 7 3GHz computers.

    When you use REDCINE (tm) to convert your R3D footage to 16 bit TIF frame files you should put the 500GB disk with the project folder into your XP (tm) computer as a slave disk and change the output directory to pipe each shot's frames into an empty shot folder in the project structure, doing things that way can save days of file copying. Also when you export from REDCINE (tm) you should adjust the output name so that the file names are padded with zeros to get my 8 number filename "Padded" format, that saves the step of renaming the files so that my programs can work with them. To do that try changing the filename control string to "00#6.E" REDCINE (tm) may not let you enter more than 6 digits for the frame number, so the two extra zeros are just text in front of the frame number, but make the leading zeros enough to work with my programs, anyway if you can get the files in the right format into the I01 folder in my programs project structure you will save many steps and a great deal of time.

    When you export the frames from REDCINE (tm) you need to adjust its trims to have the zero numbered frame the "slate" clap or beep frame, that makes syncing up the audio easer since you just trim the head off the WAV for each track so that the WAV starts with the clap or beep. The first or zero audio frame would be the clap or beep and the first or zero image frame for all resolutons would be the flash or slate close frame.

    Once you get the TIF frames into the project you can use the Insert command to generate the low res "Proxy" frames for editing and the sync sound playback. You can do the key frame grading before or after you make the "workprint" "proxy" frames, I would select a key frame and do a rough grade then touch up the grade just before making the output frames for the film recorder or DVD.

    There are still some things that will be needed for better results, one is a better resize for reduction, but you can use external freeware and such to do that for now, probably. You could use the Sharp/Soft filter before doing the reduction to avoid aliasing and noise better, but that would take some time. If the original frames are not too sharp to start with and are low noise, you can use the automatic reduction with the Quad pixel and get usable results. Enlargement works well as things are. If you are going 1:1 then there should be fewer issues.

    The speed of processing depends mostly on the speed of the disk arrays. If you read from one project folder on a RAID array and write to another RAID array you may get faster thoughput on the workflow. If you can use Solid State harddrives to hold the low res "proxy" frames for the sync playback you may be able to use higher resolution files and get fewer dropped frames. The dropped frames do not affect the high resolution output, they are just an issue if your harddisks are not fast enough for the sync playback with the Animate, Pick, and View commands. The sync on the final output should be absolute.

    What should be able to be done now with 16 bit TIF In and Out:

    1) Edit a feature film in reels with playback of the workprint res for up to 2.9 hours.

    2) Do color correction at high color depth using LUT and 64 bit float.

    3) Do rolling titles.

    4) Do fades and other optical effects such as amimated and super titles, maybe blue screen with the aid of some rotascope at 24bpp using some macro commands.

    5) Make a high quality 512 track 16 channel sound Mix for the motion picture sound track. It is 256 full length tracks plus up to 10000 times 256 shot tracks. Limited by disk space needed, many TB for a complex mix, less for something with fewer tracks and channels.

    6) Output edited color corrected frames at varous resolutions up to 6K or more, 24bpp BMP for DVD or 48bpp 16 bit TIF film recorder and other uses.

    The 24bpp limit on the composites is from the macro commands that the program has now, you make a new shot and edit that into the edit list with the other footage like an optical. As time goes on I will try to upgrade this to 48bpp, but since the frames are graded before the composite the results are usable as is.

    The system was made to work with film negative scans, but the color correction can correct any source that has some color information in it, using REDLog (tm) 16 bit TIF files seems to produce nice results from RED ONE (tm) footage.

    If you have any questions feel free to ask or contact me. If you would like me to grade some test shots for you I might be able to do that, and send you the KCC files for the Key frames you pick so that you can process all the frames from each shot on your render farm with the settings I used. You can clone the KCC files from one shot to another, that saves some time if the shots are about the same look and exposure. Right now each shot just gets one key frame for the CC, it works like a printer tape, one light change per shot, but vastly more correction with the curves, levels, gammas, and color matrix.
    Dan Hudgins is developing "Freeish" 6K+ NLE/CC/DI/MIX File based Editing for uncompressed DI, multitrack sound mixing, integrated color correction, DIY Movie film scanning, and DIY Movie filmrecorder software for Digital Cinema. RED (tm) footage can be edited 6K, 5K, 4.5K, 4K, 3K, 2K, or 1080p etc. see http://www.DANCAD3D.com/S0620200.HTM (sm) for workflow steps.
    Reply With Quote  
     

  2. #2  
    Junior Member
    Join Date
    Sep 2008
    Posts
    16
    Excellent. This is what we need for 4K post.
    Reply With Quote  
     

  3. #3 Has anyone used the "SwissKinfe" v3.22 program (for external FAT32 drives) 
    Senior Member Dan Hudgins's Avatar
    Join Date
    Feb 2007
    Location
    San Francisco, CA USA
    Posts
    1,370
    Quote Originally Posted by ThydNostrum View Post
    Excellent. This is what we need for 4K post.
    The more the merrier, my goal is to have my DI system bug free and crashless, FreeDOS FAT32 (tm) on the "render farm" may help keep things running for months without a crash, we will see. Anyway I do need help testing everything so I can work out any bugs, in general any bugs there might be are ones I do not know about, so tell me if something does not work as it should. Sometimes after an Error it is best to quit the program and reboot the computer, just in case something got messed up. XP (tm) may show more issues, if you get a "protection error" do not ignore it, just tell me exactly what you did to produce it, I might be able to fix that if I can get it to show up on my system also.

    >>Note, Some tasks or programs used require Windows ME (tm) or Windows XP Home SP3 (tm) with the FreeDOS FAT32 (tm) disk mounted as a slave drive. Things may work better if all disks are formatted FAT32, which should work up to 2TB per disk.<<

    Anyway the thing I would like to ask about today is:

    Has anyone used this program "SwissKinfe" v3.22, that claims that it can format external harddrives FAT32. Here is their link,

    http://www.compuapps.com/download/sw...Swissknife.htm

    Related to this, does anyone know of an external harddrive case that takes EIDE drives up to 500GB and larger, and has a USB driver for Windows ME (tm). I was at a computer store today, and the external USB drive cases only work with 2000/XP and Vista. Windows ME (tm) might work better for making your DIY film recorder to get high resolution VESA video modes, and being able to "Hot swap" 500GB harddrives by using USB external drives might be a way of feeding frames for printing. The film recorder would pause when a frame was not found and wait for another drive to be plugged in, and such.

    I do not know if there is another way to "HOTSWAP" 500GB EIDE harddirves under Windows ME (tm) or FreeDOS FAT32 (tm). Under XP (tm) this would not be an issue, maybe, but I need to look into some video issues under XP (tm) for the 2048x1536x32 resolution. It would be possable to stop the film recorder between groups of shots and reboot FreeDOS FAT32 (tm) with another 500GB data disk (you would not want to stop in the middle of a shot or you would get a slight flash frame), but it would be nicer to find some way to "Hot Swap" or store all the frames for each 2000 foot reel, I would rather not put splices in the film rolls. FreeDOS FAT32 (tm) seems to have Network suport so it might be possable to make a Network drive letter, and draw the frames from that Network drive, but I have not figured out how to do that yet... Any help or ideas on this running out of frames space on the film recorder computer will be helpfull.

    If you are using another film recorder (not DIY using my DANCINEL.EXE (tm)), then these frame storage aspects are not issues, since that film recorder would already have a way to load 2000 feet worth of frames?

    4096x3112x6=77MB

    77MB * 16 * 2000 = 2.5TB+overhead (for 4K (looking into how to do this))

    If 2TB EIDE come on the market and you can put four of those into a FreeDOS FAT32 machine then the film recoder should be able to have enough room for 4K or better? Right now I will have to find out how many 500GB EIDE drives one can get to work at the same time, some how under FreeDOS (tm) or Windows ME (tm).

    For "2K" output the drive space required per reel would be about,

    2048x1536x6=19MB, (16 bit TIF, 16 bit NIP103, 4:3)

    2048x1152x6=14.1MB (16 bit TIF, 16 bit NIP103, 16:9)

    19MB * 16 * 2000 = 608GB + overhead. (4:3)

    14.1MB * 16 * 2000 = 452GB + overhead (16:9)

    ---

    2048x1536x4=12.3MB (11,11,10 bit NIP203)

    2048x1152x4=9.5MB (11,11,10 bit NIP203 for 16:9 letterbox in 4:3)

    12.3MB * 16 * 2000 = 394GB + overhead (for full "2K" video board image)

    9.5MB * 16 * 2000 = 304GB + overhead (for '2K" 16:9 letter box?)

    ---

    2048x1536x3=9.5MB (24bpp BMP [supported now in DANCINEL.EXE (tm)])

    9.5MB * 16 * 2000 = 304GB + overhead

    Taking a quick look at these numbers, if I add support for my NIP203 file type to my film recorder software you may be able to put a reel of output frames for 2K on a single 500GB EIDE drive, it depends on the overhead with FAT32, I will need to try filling up a drive and see how many frames I can fit on. With letter box frames (2048x1152) the space would be less, but using squeeze frames give more scan lines in the 16:9 sound frame area (if the vertical size in the film recorder is reduced). My NIP203 may (?) hold more tone detail than a DPX file with 11 bits Red, 11 bits Green, and 10 bits Blue in 32 bits per pixel, so I should look into that. 16 bits per color would be better because of the film recorder gamma adjustments to the gamma curve, if bigger disks come out or one can workout a hotswap or network then that can also be an option.

    4K film recorder frames for a reel might fit on four 500GB drives, but this is something I will need to look into some more. I still need to find out what issues there are with SATA drives and FreeDOS FAT32 (tm)/Windows ME (tm), and if that can be made to work then 2TB drives would be at hand for 4K work...

    I tried the freeware FAT32FORMAT.EXE (tm) under XP Home SP2 (tm) and it worked fine to format a big FAT32 partition on a disk so that I can boot over the XP (tm) disk with the FreeDOS FAT32 (tm) boot CD in the CD drive (boot order CD then HD) so I can access all the files on the XP (tm) 500GB disk with both XP (tm) from HD and FreeDOS (tm) from CD. If you let XP (tm) make a NTFS partition like it wants to then you cannot boot from the FreeDOS (tm) CD and get things to work (?), here is the link to FAT32FORMAT.EXE if you need it,

    http://www.ridgecrop.demon.co.uk/ind...at32format.htm

    Anyway, I was interested to know if anyone has used "SwissKinfe" for formating external drives FAT32?
    Dan Hudgins is developing "Freeish" 6K+ NLE/CC/DI/MIX File based Editing for uncompressed DI, multitrack sound mixing, integrated color correction, DIY Movie film scanning, and DIY Movie filmrecorder software for Digital Cinema. RED (tm) footage can be edited 6K, 5K, 4.5K, 4K, 3K, 2K, or 1080p etc. see http://www.DANCAD3D.com/S0620200.HTM (sm) for workflow steps.
    Reply With Quote  
     

Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts