Red@home Workflow?

[Casey Green]

With the ongoing discussions about RED workflow, there seems to be two schools of thought.

1) Prosumer on up - REDCINE / RED Alert! Apple / Adobe / Avid, etc.
2) Higher End (Scratch, Da Vinci, Pablo, etc.)

(These can of course mix and match since there is no "right or wrong" method here and many other options not listed above.)

Either way, rendering times become a factor quickly.

This got me thinking about the time I spent working with various 3D apps / software companies over the years. Most eventually offered their customers ways to divide up the Rendering time across many CPUs by adding Network Render Farm utilities.

How great would a REDNET app be where you could take advantage of your entire Render farm or even just all of the unused cycles of CPUs on the Network? This could be an plugin for REDCINE and RED Alert!.

Also, while we're on the subject, for those that are more experimental in nature, perhaps a REDUser project could be set up...

Many of you have already heard of the SETI@home project:

"SETI@home is a scientific experiment that uses Internet-connected computers in the Search for Extraterrestrial Intelligence (SETI). You can participate by running a free program that downloads and analyzes radio telescope data." ( http://setiathome.berkeley.edu/ )

What about a prosumer version of the REDNET RenderFarm that worked just like SETI@Home, but instead, REDUsers contributed their extra CPU cycles to other's projects?

Just an idea, but could be fun and very useful for the guys who can't afford the horsepower of the big post houses yet.

[Gunleik Groven]

This is a really crazy and fun idea!!!!

It has some very practical limiting factors (ok, that was an understatement...) - bandwith for one, but the pure idea is an incredible fun thing.

Let them come!

Gunleik
(Actually out of irony by now)

[Kyle Mallory]

There are a couple of aspects to consider, but the principal one being, which is faster: to decode a single frame on a local machine, or to send a single frame of RED RAW data across the internet to another machine, let it decode the frame, and then send back the transcoded frame (which is likely going to have a larger frame size).

In the case of a 3D render farm, at the least you are taking a few kilobytes of essentially textual data that is resolution independent, and represents an entire scene, and sending that over a network to produce an image that is probably 10-100 times the input size, and anywhere from a few minutes to tens-of-hours to render a single frame.

In the case of RED you are transfering a couple megabytes of data that represents a single frame (instead of the entire scene), transcoding it to another format, which would take anywhere from a couple of seconds to a minute or two (ie, less time than the actual transfer), and then sending back a single frame that is 2-10x bigger than the original frame.

I hope you're seeing the reality here. While the idea of a REDNET is cool, the practicality of it just isn't viable. You'd spend more time streaming data out and back than it would for you to just render it yourself. On a local network, a distributed render farm has merit, but as a community tool, over the Internet....

I know, I'm a kill-joy.

[Jeff Kilgroe]

A RED render system like the SETI@Home implementation probably wouldn't work. I just don't see any of the numbers working out in favor of it, but I suppose no way to tell for sure unless someone builds it....

RED has already given us the REDLINE command line render tool for OSX with Windows and Linux versions just around the corner. I'd say that RED's own render management tool would be a cool addition for this, but there's already some pretty cheap tools to allow good quality network rendering.

Spider is free
Butterfly Net Render is dirt cheap ($375 for a 15-system farm license)

There's probably some others. But I would definitely be in favor of RED pimping us out with their own which can be used directly from REDCINE, SCRATCH or eventually any NLE or post software that ends up supporting REDCODE and the need to process it.

[Casey Green]

...I would definitely be in favor of RED pimping us out with their own which can be used directly from REDCINE, SCRATCH or eventually any NLE or post software that ends up supporting REDCODE and the need to process it.

Yes, the RED@Home may or may not be practical, but the other idea I had about a REDNET app for a private Network Render Farm might really be desired.

Imagine how much faster your projects would get done with 4, 8, or 16+ 8 core Mac Pros churning away REDCODE.

[Kyle Mallory]

Another thing you have to consider, even in a local network environment, is "what are you rendering?"

Are you rendering edits, composites, colors and conforms from your post suite? Or, are you simply rendering a frame of R3D to a DPX, or a DPX to an R3D (ie, REDCINE)?

The first isn't a function of RED or anything having to do with RED. It's whatever tools your using to do those things. I believe this is where the biggest potential lies in distributed rendering. Sure a fair amount of time is spent decoding/encoding frames, but I suspect the real performance hit comes from the image processing on those frames post-decoding/pre-encoding.

The second case, distributing the actual encoding/decoding is probably something that would have to be build into the codec/command-line tools. Of course, you could use REDLINE to decode 30 frames on one CPU, and the next 30 frames on another CPU, but this would probably be the practical limits of such a render farm.

Once linux tools come out, I'll probably spend some time building a basic multi-node render farm to address the second case (unless RED beats me to it), but IMO, the real benefit of multi-node rendering comes in the integration of the rendering into the post suite.

[Joe Carney]

How about porting RedCine rendering to a PS3 running Linux.
For the fold@home project, they outperformed a desktop PC by
at least a factor of 10. They're used as affordable supercomputer clusters at some universities. Now that would rattle Sonys' cage,:turned:

[Jeff Kilgroe]

Imagine how much faster your projects would get done with 4, 8, or 16+ 8 core Mac Pros churning away REDCODE.

That was my point with the mention of other render manager software like Butterfly Net Render. You can do this today with RED's REDLINE utility and a render manager like that.

When RED releases windows and linux versions of REDLINE, then it will be even easier. I already have a small render farm here for my 3D animation work. As soon as the Windows command line tool (or Linux, as my nodes are dual boot) is available, I can render away.

RED having their own render manager would be icing on the cake, but they would would just be re-inventing the wheel.

[Radoslav Karapetkov]

Cool idea!

The whole thing could be encrypted and p2p based.

But I guess it would be very difficult to make working software\network for this.

[Gunleik Groven]

Maybe if the frames could be split up in a torrent-like way.... But you DO have a major bottleneck in your internet bandwith.... LOL

But I really like the concept.

Gunleik

[Casey Green]

Perhaps as bandwidth increases, this will no longer be a problem.

I like the idea of RED building this ability into their apps, but to be able to use REDLINE with other existing node software is great.

But who knows what the RED pixies might come up with that hasn't been accomplished before in this area. :sorcerer:

[AmariahOlson]

Now this is a ROCKING IDEA!! I am in and we have about 20machines you guys can use when we are not!~

[Radoslav Karapetkov]

Hey, what about something like this:

http://scarletuser.com/showthread.php?p=31024#post31024

[Andrew McCarrick]

Instead of sending single frames, why not whole seconds of images wrapped in some kind of render format. Send out 24 frames to a computer, takes a couple minutes and it starts sending back data while its still processing the rest.

So you hit the render button ---> Divides number of seconds over available computers---> Render Computer recieves render request ----> Render Computer starts processing ---> Frame 1 returns (once processed) to sender while Frame 2 starts being processed.

So it's not quiet real time processing, but it be quicker, because you have multiple computers working on multiple seconds at one time.

Basically the Render Computer downloads a whole second, before it starts processing, but spits back individual frames in real time. Or it could go the other way around (Recieves individual frames, feeds back a whole second), or both could be wrapped in a master file, so it's a download (of a whole second) then an upload (of a whole second).

Kind of like a combination of Napster/BearShare/Kazaa and the @home system.