View Full Version : How much horsepower needed to edit 4k?
Michael Struthers
01-04-2007, 10:00 PM
Will a new maxed-out imac do it?
Stephen Gentle
01-04-2007, 10:07 PM
I think that a Mac Pro would probably be the minimum.
Emanuel A.
01-04-2007, 11:18 PM
Here is some info related:
http://www.reduser.net/forum/showpost.php?p=390&postcount=10
http://www.reduser.net/forum/showpost.php?p=704&postcount=16
Nick Shaw
01-05-2007, 03:29 AM
Since RedCine uses the GPU to do some of it's processing, I would think the graphics card in an iMac may not be up to the job.
Nick
Jeff Kilgroe
01-05-2007, 08:10 AM
Since RedCine uses the GPU to do some of it's processing, I would think the graphics card in an iMac may not be up to the job.
Nick
LOL!
I guess that means my MacMini is out for an editing system. :D
Seriously, I think that for editing 2K or 4K a current quad-core Mac or PC can pull it off just fine. The real big issue is having a drive array that can sustain at least two streams of whatever format you're working in. And this can be alleviated somewhat by proxy editing and keeping full-res footage offline for much of the process. I currently do all my editing at 1920x1080p and typically 24fps. That's my target for all my animation rendering right now, but I'll bump that to full 2K or even to 4K once I get my hands on RED. ...I'll have to buy more render nodes to handle 4K -- this RED thing sure is getting expensive. I'm using G5 quads right now as well as various dual core and a couple quad-core PC systems in my workflow now. I plan to buy a couple new Mac Pro systems this spring and I'm hoping the 8-core units with the upcoming Stoakley chipset are ready when I am. I'll be building one, maybe two 8-core PC systems once Stoakley is shipping. I plan to keep the G5 quads in my workflow, but will demote my current dual and quad-core PC systems to render farm duty.
And I realize this post was probably not that helpful... I guess we'll know more in the coming weeks/months about requirements for RedCine and editing what we shoot with RED. But I wouldn't cross that new 8-core Mac or PC with 12GB RAM and 4x750GB RAID off your shopping list just yet.
I'm expecting 8-core Macs to be announced anytime between MWSF in a few more days and no later than NAB. Availability should be NAB or shortly after if Apple waits for Stoakley, which they may very well do.
Jeff Kilgroe
01-05-2007, 08:15 AM
I guess if RedCine uses the GPU, we should also plan for buying a decent video card. ...I would anyway since I need it for my graphics work as it is. I just wish Apple would get off their lazy rear end and start offering some decent video options. For systems geared toward the creative / graphics professional, the X1900XT is a little weak and the FX4500 is outdated and overpriced.
NOMADIC
01-05-2007, 10:41 AM
my guess is an imac is going to be crying if you try running 4k footage on it, the other way is just capture photo jpeg, edit on that, it sucks, then let it export with the 4k footage from the timecodes. it might take mannnyyy hours, but whatever, go catch up on that dvd box set of the gong show you've got.
Chris Kenny
01-05-2007, 04:24 PM
I guess if RedCine uses the GPU, we should also plan for buying a decent video card. ...I would anyway since I need it for my graphics work as it is. I just wish Apple would get off their lazy rear end and start offering some decent video options. For systems geared toward the creative / graphics professional, the X1900XT is a little weak and the FX4500 is outdated and overpriced.
Of course, it's very unlikely those will still be the options by the time Red is released.
Seriously, anyone who's buying a computer specifically to edit Red footage should wait as long as possible; you'll end up getting more for your money.
ericyoung
01-05-2007, 04:31 PM
How multiprocessor multithread savvy is Redcine? Will it scale well with more processors, hyperthreading etc?
Brian Ferguson
01-05-2007, 07:19 PM
my guess is an imac is going to be crying if you try running 4k footage on it, the other way is just capture photo jpeg, edit on that, it sucks, then let it export with the 4k footage from the timecodes. it might take mannnyyy hours, but whatever, go catch up on that dvd box set of the gong show you've got.
I know you are kidding. I can't run a half resolution 1080 (540x960) video podcast on my Mac Mini at home. The guy at the Apple store told me you can't run FCP on a Mac Book because of the lack of dedicated video ram (shared memory) so I bought a MacBook Pro. So that might give you an idea of scale. I have an iBook and had a hard time opening RAW files from my Canon 20D in iPhoto, took a long time.
My sentiments are echoed throughout this thread regarding the GPU.
The GPU's role in modern computing is becoming very central. More and more we see clever programmers (like Mr.Natress) harnessing the power of hardwired gpu functions, to which I believe the role of the Central Processing Unit as we know it will completely change in the not too distant future. Forget clockspeeds.. forget cores. We've seen IBM's cell processor do things on the playstation 3 that would make a conventional CPU look like a pocket calculator. To get a general Idea, PS3's cell produces 16 TRILLION calculations per second, a PC or Mac does a few billion. Now if you could have that inside your editing suite, it would be, well.. sweet.
So.. going off on a tangent, what was my point... Ah..yes..
The future of computing is EXCITING. In a couple of years, be prepared to forget the notion of cores and clock speed.
For now, a good video card would be a very good idea. On most good PC motherboards, you have a Dual-SLI slot, which allows you to bridge two video cards so that they function as a single unit. This also can bring about massive performance gains in my opinion (Someone correct me if this is not the case).
Sanjin Jukic
01-06-2007, 03:02 AM
I wish that Mac OS X with Leopard release finally would get driver(s) to run the NVIDIA Quadro® Plex 1000 visual computing system (VCS).
512MB - 1GB Frame Buffer per GPU
Uncompressed 8-, 10-, or 12-Bit SDI Output
2K 2048x1080i from 23.976 Hz to 60.00 Hz (SMPTE372)
http://www.nvidia.com/page/quadroplex.html
Rob Lohman
01-06-2007, 03:29 AM
How multiprocessor multithread savvy is Redcine? Will it scale well with more processors, hyperthreading etc?
Both the codec & REDCINE are fully multi-processor/core aware and utilize it as best as they can. Hyperthreading is sort of a relic from the past and I don't expect it to give much benefit with stuff like this. You will definitely want a multi-processor/core machine (more specific numbers will be available later).
Jeff Kilgroe
01-06-2007, 10:56 AM
Forget clockspeeds.. forget cores. We've seen IBM's cell processor do things on the playstation 3 that would make a conventional CPU look like a pocket calculator. To get a general Idea, PS3's cell produces 16 TRILLION calculations per second, a PC or Mac does a few billion. Now if you could have that inside your editing suite, it would be, well.. sweet.
Whoa, there... lay off the Sony kool-aid. I've personally worked with Cell and I'm working with it right now on an embedded control systems project. I knnow what it can and can't do. Cell is a very nice CPU, but in reality it's just a PowerPC 907 series that has all the legacy junk stripped off so it's not backwards compatible with anything. It's trimmed and lean and mean and has some nice abilities like running individual cores at arbitrary speeds. More cores (like 3, 7 and 9 core variants of Cell) are easily created on the CPU die because it's trimmed down and with their current 65nm process, they can fit more of these smaller CPU cores on a single chip. Intel won't hit unified quad-core and 8, 12 and 16 core CPUs until they transition x86 to 45nm later this year.
As for what PS3 can do, the true power of PS3 is not the Cell processor. We heard all this same type of crap the last time around with PS2 and the EmotionEngine CPU - (hybrid MIPS R5400). You can chalk up the Trillions of operations per second claim with the PS3/Cell right along with the "it's illegal to export PSX and PS2 to the middle east because they're classified as supercomputers" rumor. Sony is making the trillions of operations per second claim based on the number of cores * number of possible pipeline steps * MHz. It's a bogus claim, but there is no industry standard that specifically declares what an "operation" is. Perhaps I've spent too much time in game development and I'm glad I don't do it anymore, but we have to look at the hardware for what it is... In the PS3's case, it is first and foremost a game system. It is designed to play games and it does that rather well -- although the current crop of games are junk. If Cell were half as revolutionary as the Sony hype juggernaut makes it out to be, then people would be tossing x86, Itanium/EPIC and just about every other major CPU architecture in the trash in favor of Cell.
The real power of the PS3 is the nVidia GPU, which has been tweaked to work in tandem with Cell, the onboard memory architecture and common HD video resolutions. Additionally, game developers can specifically optimize code for the PS3 platform and it allows them more freedom to create visually impressive games than it does with current PC hardware due to no need for maintaining compatibility with countless hardware configurations. 6 months from now, PC games will look far better than PS3 games - just as what happened with the PC vs. PS2 and XBOX. IMO, there's a few PC games right now that are on par with what PS3 is showing.
And let's not forget all those revolutionary Cell-based products and turn-key systems that Sony has been bragging about for the last three years. None of which have come to market, let alone have even been demonstrated with any amount of reassurance that they will come to market. Remember, we're talking about the same Sony that was busted at CES in '99 for hyping up PS2 and EmotionEngine by showing a demo of a real-time rendered ball covered with over 2 million interactive hair fibers. ...When I say they got busted, they had to fess up and admit to the demo running on an SGI Onyx2 with dual RealityEngine graphics pipes and 4xR10K 200MHz CPUs (think $80K computer at the time) was tucked under a table behind the booth. And then they had the brass cajones to claim it was representative of what the EE/PS2 could do when released. Even though the EE was nothing more than an R5400 CPU with partially integrated nVidia graphics chip.
OK, I'm off my Sony rant... Sorry everybody. But their CE division and PlayStation / Media marketing has no shame.
Jeff Kilgroe
01-06-2007, 11:05 AM
Both the codec & REDCINE are fully multi-processor/core aware and utilize it as best as they can. Hyperthreading is sort of a relic from the past and I don't expect it to give much benefit with stuff like this. You will definitely want a multi-processor/core machine (more specific numbers will be available later).
By fully multi-processor/core aware, can you confirm that it will scale to 8 CPU cores or beyond to 12 or 16? I'm not trying to be too skeptical or put you guys on the spot, but I deal with "multithreaded" software on a daily basis and see these claims every day. Yet, very few applications can do what they claim and I'm tired of dealing with "fully multithreaded" software that does OK with 2 cores, barely does any better with 4 and crashes into a flaming pile of bits when given 8 CPUs to work with.
Anyway, I'll leave it at that because I have high hopes for RedCine and the software/tools you guys are creating. I've done my fair share of multithreaded software development and I know it's not an easy task. Please don't let us down! Because, sadly, most other software developers are really struggling with this one and have for some time. eg: MentalRay = good multithreaded implementation, but only scales to 8 CPUs per node. Lightwave|3D = bad multithreaded implementation - up to 16 threads, but poor scheduling some core components not even multithreaded.
Jeff Kilgroe
01-06-2007, 11:16 AM
I wish that Mac OS X with Leopard release finally would get driver(s) to run the NVIDIA Quadro® Plex 1000 visual computing system (VCS).
Hopefully! Although the QuadroPlex is a bit on the expensive side. I like it though -- it's a total flashback to the SGI days with the external RE2 rackmount graphics pipes. :D
Although, without the Mac having at least dual PCI x16 interfaces (hopefully the new Mac Pro w/8 cores will hae this), the Quadro Plex would be a bit limited on bandwidth to/from the system.
I just hope Apple updates their graphics offerings by NAB to at least have Quadro 5500 and hopefully some good mid range offerings like the GeForce 8800/8900 series. Dual interfaces / SLI is a must, IMO and I'm still shocked that the Mac Pro didn't ship with two linked x16 slots.
Sanjin Jukic
01-06-2007, 12:52 PM
Hopefully! Although the QuadroPlex is a bit on the expensive side. I like it though -- it's a total flashback to the SGI days with the external RE2 rackmount graphics pipes. :D
Although, without the Mac having at least dual PCI x16 interfaces (hopefully the new Mac Pro w/8 cores will hae this), the Quadro Plex would be a bit limited on bandwidth to/from the system.
I just hope Apple updates their graphics offerings by NAB to at least have Quadro 5500 and hopefully some good mid range offerings like the GeForce 8800/8900 series. Dual interfaces / SLI is a must, IMO and I'm still shocked that the Mac Pro didn't ship with two linked x16 slots.
Only we can wait for Tuesday MWSF Apple hardware announcements in a keynote speech from Steve Jobs and til than read what the Mac rumor site wrote:
"- Standard graphics bumped to nVIDIA 7600GT; high end options include ATi X1950 XT, nVIDIA QuadroFX 4500, and a new high-end professional card from ATi as yet unnamed.
- SLI and CrossFire supported cards will also be included, presuming both technologies are actually supported. A build to order SLI configuration based around the nVIDIA 7950 GX2 has been previewed to Apple partners but it is not known exactly what cards will be offered in the final product."
http://www.macosrumors.com/20061218-1A.html
Rob Lohman
01-06-2007, 01:48 PM
By fully multi-processor/core aware, can you confirm that it will scale to 8 CPU cores or beyond to 12 or 16? I'm not trying to be too skeptical or put you guys on the spot, but I deal with "multithreaded" software on a daily basis and see these claims every day. Yet, very few applications can do what they claim and I'm tired of dealing with "fully multithreaded" software that does OK with 2 cores, barely does any better with 4 and crashes into a flaming pile of bits when given 8 CPUs to work with.
REDCODE benefits more from multiple CPU's than REDCINE (which uses the GPU as well, hence less work for the CPU's).
On a 4 cpu machine here REDCODE is giving me around 95% CPU load on all the 4 cpu's. I'm gonna test 8 and hopefully 16 cpu machine's over the next few months. It should scale well to 8 cpu's, after that I expect a slight drop off in CPU utilization. But we'll have to wait and see :)
I understand you're concern and we are really focussed on using the available resources well (helps us in decoding speed, so that's good for everyone!)
Jeff Kilgroe
01-06-2007, 02:50 PM
Rob, thanks for the response. That's exactly what I wanted to hear (er.. read). :)
Don Woods
01-06-2007, 03:00 PM
Thanks Rob that is good to know. I really think that to have a smoth running red system you are going to need to look at Mac Pro all the way. And even mabye a render server situation (if we REDCINE can). As for just editing if you are using REDCODE. I don't think it would be to frusterating to run it on a Mac Book Pro. At least I hope not. I use high rez RAW files all the time in Apature on mine and it kicks but. So I think we are good. But for actually processing the footage the more power you got the faster the blue (RED) bar will move.
Hey Rob I don't remeber if you or anyone stated on DVXuser. But is REDCINE going to be able to acess a cluster of servers and utiliz them for processing...
Jeff Kilgroe
01-06-2007, 03:53 PM
Hey Rob I don't remeber if you or anyone stated on DVXuser. But is REDCINE going to be able to acess a cluster of servers and utiliz them for processing...
That was my very next question. Please say I can use my render farm... Puhleeeze. ;)
Whoa, there... lay off the Sony kool-aid. I've personally worked with Cell and I'm working with it right now on an embedded control systems project. I knnow what it can and can't do. Cell is a very nice CPU, but in reality it's just a PowerPC 907 series that has all the legacy junk stripped off so it's not backwards compatible with anything. It's trimmed and lean and mean and has some nice abilities like running individual cores at arbitrary speeds. More cores (like 3, 7 and 9 core variants of Cell) are easily created on the CPU die because it's trimmed down and with their current 65nm process, they can fit more of these smaller CPU cores on a single chip. Intel won't hit unified quad-core and 8, 12 and 16 core CPUs until they transition x86 to 45nm later this year.
As for what PS3 can do, the true power of PS3 is not the Cell processor. We heard all this same type of crap the last time around with PS2 and the EmotionEngine CPU - (hybrid MIPS R5400). You can chalk up the Trillions of operations per second claim with the PS3/Cell right along with the "it's illegal to export PSX and PS2 to the middle east because they're classified as supercomputers" rumor. Sony is making the trillions of operations per second claim based on the number of cores * number of possible pipeline steps * MHz. It's a bogus claim, but there is no industry standard that specifically declares what an "operation" is. Perhaps I've spent too much time in game development and I'm glad I don't do it anymore, but we have to look at the hardware for what it is... In the PS3's case, it is first and foremost a game system. It is designed to play games and it does that rather well -- although the current crop of games are junk. If Cell were half as revolutionary as the Sony hype juggernaut makes it out to be, then people would be tossing x86, Itanium/EPIC and just about every other major CPU architecture in the trash in favor of Cell.
The real power of the PS3 is the nVidia GPU, which has been tweaked to work in tandem with Cell, the onboard memory architecture and common HD video resolutions. Additionally, game developers can specifically optimize code for the PS3 platform and it allows them more freedom to create visually impressive games than it does with current PC hardware due to no need for maintaining compatibility with countless hardware configurations. 6 months from now, PC games will look far better than PS3 games - just as what happened with the PC vs. PS2 and XBOX. IMO, there's a few PC games right now that are on par with what PS3 is showing.
And let's not forget all those revolutionary Cell-based products and turn-key systems that Sony has been bragging about for the last three years. None of which have come to market, let alone have even been demonstrated with any amount of reassurance that they will come to market. Remember, we're talking about the same Sony that was busted at CES in '99 for hyping up PS2 and EmotionEngine by showing a demo of a real-time rendered ball covered with over 2 million interactive hair fibers. ...When I say they got busted, they had to fess up and admit to the demo running on an SGI Onyx2 with dual RealityEngine graphics pipes and 4xR10K 200MHz CPUs (think $80K computer at the time) was tucked under a table behind the booth. And then they had the brass cajones to claim it was representative of what the EE/PS2 could do when released. Even though the EE was nothing more than an R5400 CPU with partially integrated nVidia graphics chip.
OK, I'm off my Sony rant... Sorry everybody. But their CE division and PlayStation / Media marketing has no shame.
*Puts the koolaid down*. I really dont know much about the PS3 but thanks for illustrating the way it works. But I am glad to know that NVIDIA was the hero in this one. Which is why you may have noticed that my rant was about GPU's in the first place.
The point I was trying to make is that there is a paradigm shift from raw processing power to clever processing power. Why use a "virtual" processor so to speak when you can have the most common functions you need already hardwired ready to go as in the case of the GPU. My point is that undeniably, this is where computing is heading. Cell or Not.
Rob Lohman
01-07-2007, 02:57 AM
REDCINE will not support a render farm, we may have some other options but can't say much about that at this point in time.
If you have multiple machines split footage (clips) over these machines and have each process a part of what you shot. I know this involves a bit more work on the user's part, but it does give you more control (and allows you to generate QuickTime files which is not possible in a render farm setup).
On editing with REDCODE, keep in mind that it's not just huge resolution RAW file. That would be easy. Wavelet decompression is expensive to do (cpu wise) and at these high levels debayer time starts to really make an impact as well.
Jeff Kilgroe
01-07-2007, 07:41 AM
The point I was trying to make is that there is a paradigm shift from raw processing power to clever processing power. Why use a "virtual" processor so to speak when you can have the most common functions you need already hardwired ready to go as in the case of the GPU. My point is that undeniably, this is where computing is heading. Cell or Not.
Yep. There's been lots of talk within the industry for years about using GPU functions to aid in intense processing tasks. Over the years, several have tried it, but few have succeeded in making something worth-while. But I think GPU power is finally at a stage where it is truly powerful. The texture/image processing functions on most current GPUs are incredibly powerful and they offer abilities in real-time that take even the latest CPUs several minutes or even hours to render. This is why we're now seeing advanced GPU functions being utilized in upcoming OS releases from MS and Apple. The GPU is no longer relegated to "that chip on your video card which helps you play cool games". It is another processor in your computer that deserves equal attention and should share the same status as your CPU because in many ways it's just as powerful, in some ways more and should be used for far more than pumping texture polygons into the latest version of Quake.
As for Cell, I don't mean to bash it, it's a pretty good product and can fit so many uses a lot better than x86 chips and Cell is much more powerful than ARM or older Motorola and MIPS chips commonly used sor such things. Think compact, streamlined embedded applications. Cell makes perfect sense for autonomous applications for manufacturing systems and robotics. Think robotic arms in atuo assembly lines where each motor/joint of the arm has it's own "brain". Cell's biggest strength is how easy it is to scale and link collections of the processor to form larger node-based CPU clusters. Hence its name... If Cell continues to develop and follow IBM's intended path, you won't see it become the CPU in a desktop workstation. But that new 32-core Mac pro you buy in 2012 could very well have various forms of Cell processors in it to control secondary device functions like RAID subsystems, coordinate external interfaces like multiple FW800 busses, etc..
The reason why the Sony hype machine uses Cell to promote PS3 is because it's a cool sounding new thing that most people are unfamiliar with. It's all marketing... They can make it out to be what they want people to think it is very easily. They can't hype the exclusive nVidia GPU, even though that's pretty cool in itself. Why? Every gamer knows that the nVidia chip, which Sony has exclusive rights to today, is nothing more than a $300 PC video card within the next two months. And there will also be a $550 model at that time that's nearly twice as good. Actually this has already happened. We're just waiting for PC software to catch up to GPU tech...
Jeff Kilgroe
01-07-2007, 07:56 AM
REDCINE will not support a render farm, we may have some other options but can't say much about that at this point in time.
Hmmm... OK. The anticipation over RED and how you guys are going to make this all work is killing me. :)
If you have multiple machines split footage (clips) over these machines and have each process a part of what you shot. I know this involves a bit more work on the user's part, but it does give you more control (and allows you to generate QuickTime files which is not possible in a render farm setup).
...And such is often the workflow for larger animation projects. Can't just dump everything to the render farm, so I'm already used to this sort of thing. As for building QT files on a render farm, I know it's not possible to have multiple render node processes trying to build on the same QT file. But it's possible to build QT and other video files in a farm setup. I have it working here with software I wrote. It's actually nothing special, just one CPU or render node will start to write the desired video file from the rendered frames as they come in. I think there's a few freeware utilities out there that do the same thing. :D
Hmmm... Now I wonder how REDcine would perform in a cluster environment.
On editing with REDCODE, keep in mind that it's not just huge resolution RAW file. That would be easy. Wavelet decompression is expensive to do (cpu wise) and at these high levels debayer time starts to really make an impact as well.
Yep. I'm aware of that... And in an editing scenario, it's all about what the editing workstation can handle. Hence my questions of support for multiple CPUs. That 8-core, stoakley chipset Mac Pro can't get here fast enough. :cool:
Don Woods
01-07-2007, 10:54 AM
Well Rob that brings up the question.. How dose apple's compressior do it? I have it set up across an xserve cluster and it makes me Quicktime files when its done... Is it not really using the cluster? If so I think I found out how to pay for my camera...
Rob Lohman
01-08-2007, 03:41 AM
As indicated by Jeff it's certainly possible, but more difficult (to implement).
I was a bit quick to say it's not possible. However, to make that work we'd have to dig deep into QuickTime to generate such files and I'm not sure that's worth it.
Whether or not compressor is using your xserve cluster for QuickTime generation I don't know. Shouldn't be too hard to find out through activity monitor and some network monitoring tools.
Rest assured we are looking into all kinds of directions & technologies.
Nick Shaw
01-08-2007, 05:09 AM
I think it's probably relatively easy to do with CBR codecs, as you can generate a placeholder Quicktime of black frames with the correct duration and frame size, then the frames can just be overwritten with the rendered frames as they are generated. Obviously with a VBR codec (such as REDCODE) it can't work the same way. That method could however potentially be used by a (future) networked version of REDCINE to generate editing proxys using a codec like DV or DVCPro HD.
Nick
EDIT: I say "relatively easy" but I am talking from a non programmer's point of view about concepts, not implementation. My last serious bit of programming was in Z80 assembler for my Sinclair Spectrum, so what do I know?!
Don Woods
01-08-2007, 09:21 AM
Hey thanks Rob.. I checked and it its utilizing the Xserves so for now I'll keep them. The question is right now you are not looking a supporting this feature and maybe in the futer you would. I am just wondering if we are going to see allot of gain. AKA the RED bar moving faster in processing images. I aske this beacuse I know when we shoot features and for two company's we work for all the time. They want dailys the next day or day of if possible. If we are shooting all day on two RED cameras. I wonder if it would be extremely benifital to send that footage through a "Render Farm" vs just one machine... What do you think... And even for final render of the project that could be going out say DPX would then it be easier..
Nick Shaw
01-08-2007, 09:27 AM
If you have multiple machines split footage (clips) over these machines and have each process a part of what you shot.
How will this be affected by any licensing that may be used by REDCINE? Will the license that we get with the camera permit us to install the software on multiple machines to divide up tasks? I would hope that at the very least it will permit use on one MacBook Pro for use in the field, and one Mac Pro in the edit suite, but multiple machines would obviously be very useful.
Nick
Jeff Kilgroe
01-08-2007, 09:28 AM
There are several ways to write video files with frames or intermediate data coming from multiple sources. Most 3D rendering applications out there will allow fro some forms of video files to be written from their node-based render modules. However, I almost always make it a practice not to do it -- one node crashes and it just throws everything else off... Whereas rendering individual frames is much more reliable and fault-tolerant. As for compressor, the primary application instance still takes care of writing the QT file when in a clustered configuration. It assembles it as the data from the other cluster nodes comes in and the next piece in order is available. It's not a huge trick to do this, that's how just about every other node-based program that encodes video files works. All it can do is wait for each frame or next sequential piece of data to come in and then tack it to the end of the video file. For complex encoding tasks such as MPEG2 compression in Compressor or other apps, often data between cluster nodes must be compared and averaged before resulting data can be written to the file. Once again, not a huge deal, but as data sizes, clip duration and cluster sizes grow, it becomes increasingly less efficeint.
Don Woods
01-08-2007, 10:59 AM
Thanks Jeff that is helpfull.. So I guess the best configuration would be to have several different machines running processing different clips and dumping them onto one RAID (or several for back up). I guess if REDCINE has some kind of batch processing where we can set a number of clips and Queue them up so the next clip starts I don't see this to be an issue, If indeed a cluster type situation is less effecient.
Rob Lohman
01-08-2007, 11:17 AM
Indeed, for now use multiple machines with REDCINE. We are still working out licensing for REDCINE and so on. We do want you to use your footage so even if you do need additional licenses they will be in line with the rest of the pricing.
Jeff Kilgroe
01-08-2007, 11:31 AM
A cluster or farm of nodes isn't necessarily less efficeint, but that depnds on the work being done. Where nodal compute configurations lose their efficeincy is in transaction speed and bandwidth. If you're spending more time transferring data back and forth between nodes than actually computing results, you're spinning your wheels. This can happen pretty easily when trying to have separate nodes generate data which depends on data generated by other nodes. It is far more efficient to have the nodes work in a linear manner. All nodes are presented with the core data set such as a RAW video clip or 3D animation scene. Then individual nodes can be instructed to begin processing a single frame or small linear sequence of frames. They complete, deposit the frames to the target storage volume and request new instructions from the host node. It's fast, it's efficient. If nodes are suddenly processing intermediate data for long GOP or IFrame video formats where they have to await full frames from other nodes and quantize their data produced with that of other nodes, we start trafficking large volumes of data. There's ways to combat this effect too when processing large data sets. Such as staggered processes where different nodes handle different render phases, etc..
I'm assuming that REDCINE has some form of batch processing, it would have to.. Or if nothing else, a solid command-line interface so that it can be adapted to various batch / render / workflow managers like Chalice, Spider, AlienBrain, etc..
Anyway, I trust Rob and the rest of the RED team to deliver something that will do what most people need. As with any new software, there should be more features and abilities to come with future revisions. I'm sure there will be requests for features which weren't obvious during development, but make perfect sense once the camera is in the hands of users. For now, we don't know much and we just have to wait and see what NAB and the release of RED will bring us.
Don Woods
01-08-2007, 01:33 PM
Very True Jeff. I think that Rob and the guys at read are doing a fantastic job and REDCINE will be a top of the line product and all this talk has proably already been discussed numbers of times in the RED war room.
Thanks ROB, one last question if you can answer it. In Jeffs post he mentions having a node based farm it would be more efficient to have the nodes work in a linear manner. Will REDCINE be able to controll a host node and tell the other nodes to render and drop the footage on a Raid...
Rob Lohman
01-08-2007, 01:55 PM
No, the current version of REDCINE only knows about itself and nothing else. We'll see what the future brings.
Mark L. Pederson
01-08-2007, 07:21 PM
No, the current version of REDCINE only knows about itself and nothing else. We'll see what the future brings.
XGRID support would awesome - a few 8-core Xserves in a cluster - chomping on 4K files like potato chips ...
Don Woods
01-08-2007, 09:07 PM
Thanks Rob for the insight.
Yes the future is looking brighter and brighter everyday and I think you guys are doing a killer job. And just like every new production pipeline. You just find the best way to work with it and make it work. REDCINE no dought will grow and different methods will be thought of and that is all part of the process.
Don Woods
01-08-2007, 09:07 PM
XGRID support would awesome - a few 8-core Xserves in a cluster - chomping on 4K files like potato chips ...
I think it will happen... REDCINE is just waiting to bust out of the box..
Emanuel A.
01-08-2007, 10:43 PM
Here is some info related:
http://www.reduser.net/forum/showpost.php?p=390&postcount=10
http://www.reduser.net/forum/showpost.php?p=704&postcount=16http://www.reduser.net/forum/showpost.php?p=390&postcount=10
is 2k too much for an at home online session?
or will it pretty much have to be 1080 if u wanna do it yourself?Onlining 2k assets at home, like its bigger brother, 4k, don't come cheap. Even getting a good 1080p online at home is a big deal. You can easily run up thousands of $$$ before cutting a single frame of media. Sh*#@ don't come cheap! My best guess is that most non to average techies like me will be offlining @ 480 and exporting for a professional 1080p posting, just like Jaime outlined above:D$5,000 for 4K online editing is the David Newman's estimated figure:
«We just demonstrated real-time online 4K RAW editing (dual stream dissolve with Dalsa Origin footage) to Adobe management under Premiere Pro 2.0 running on a Dual core dual proc Xeon Woodcrest system (about $3K PC with two drives in RAID-0.) So it can be done today, although we aren't shipping this technology yet, but we intend this be available for Red customers as the alternative workflow (for those who can't stand offline workflows.) This a simple extension of the now shipping CineForm RAW workflow. So even with the purchase of our software the system costs will be doable at $5k for a Red online.»
LINK (http://www.dvxuser.com/V6/showpost.php?p=713712&postcount=12)
FOR MORE INFO >> CLICK HERE (http://www.dvxuser.com/V6/showthread.php?t=73448)
http://www.reduser.net/forum/showpost.php?p=704&postcount=16
1) Two 4K video streams of CineForm RAW can be played on a dual proc dual core Intel Woodcrest -- quad core may do better (note: our 3Ghz Woodcrest, was slightly out-performing a 2.4Ghz dual proc quad core.) With half res'ing wavelet tricks more real-time layers will be supported.
2) Color correction, transitions, titles, etc in RT. The basics are in realtime.
3) Around 1-2 frames per second on a Woodcrest Intel system -- this lowish render speed is due to the demosaic required to convert form RAW to RGB.
4) No real need. You source is compressed (whether REDCODE or CineForm RAW) and for post renders to CineForm Intermediate this can have as little compression as the user desires. If you wish to decompress to DPX then conform those, the quality is not visibly different even for extreme workflows.
David Newman
CTO, CineForm
blog : cineform.blogspot.com
web : www.cineform.com