PDA

View Full Version : Piranha (Linux) & R3D Support



Kyle Mallory
01-01-2009, 12:21 PM
I decided to start a new thread for this. Any questions about Piranha and REDCODE/R3D, I'd encourage posting here for now.

Just a small update on my progress with native R3D support for Piranha.

Jan 1, 2009: Audio is now functional. Support for ILM/EXR HALF types, RGB (8bit), and SHORT (16bit). Currently, SHORT is recommended for performance reasons. Multi-threaded Multi-frame decoding is now supported. R3D SDK support multi-threading for decoding the current frame. The R3D plugin for Piranha will now additionally decode multiple frames at once when requested by Piranha (this requires A LOT of memory to store multiple 4k frames in the frame buffer). Currently, on my development environment (a dual-core laptop), I am seeing 24fps from 4k media at 1/8 res, and 48fps from 2k media at 1/8 res. I am working together with IFX on the metadata interface, and to improve browsing in the clip-bin (this is requiring feature enhancements to Piranha directly). Next on my list is the UI/interface for changing metadata.

Jan 11, 2009: Added "One-Light" plugin for applying user-specified Image Processing Settings via the RED SDK. It is important to clarify, as there are many plugins that support various color correction techniques, but all are applied AFTER the frame is decoded and debayered, using Piranha or other external techniques. The One-Light essentially overrides the R3D metadata and 'develops' the R3D footage according to the values specified in the plugin, before it enters the video frame enters the Piranha pipeline. Users can still apply Piranha and other 3rd Party color tools further down the chain. Because One-Light is a Piranha VFX plugin is supports all of the standard capabilities of Piranha plugins, including the ability to keyframe values, such as exposure, or white balance. For example, a traveling shot that moves from Tungsten to Daylight, can now be one-lit properly, keyframing the values and letting the REDCODE plugin transition the white balance with the camera move. Additionally, REDCODE One-Light supports Piranha's Effect Sharing. This allows a one-light to be done on all clips in a file by sharing a single One-Light instance across all R3D clips. Changing white balance values in one clip can affect all others.

There are a few significant bugs to iron out, but I hope to have some screen casts showing the plugin functionality with a week or two (nevermind the week-long trip to DC on Wednesday). Around the same time, I hope to have a beta product available to people interested in evaluating it.

Feb 3, 2009: Mostly fine-tuning some stuff, improving memory footprint, and optimizations. The plugin is pretty stable, and usable, but it does exhibit some quirks that will hopefully be addressed in the near future. Development has slowed some, in part because I'm using it, and in part because of other distractions. As promised, below is the first of a series of screencasts. These are very rough, but prove the concept. I am looking for a very high-end machine to benchmark and further optimize development on. I'm hitting the top-end of my workstation's capabilities (4-year-old hardware). If you would are interested, and have a spare octa-core laying around somewhere with a nice NVIDIA card, and want to let me borrow it for a 6-8 weeks, that would be swell.

Screencasts:
http://www.vimeo.com/3074461

JanneJansson
01-01-2009, 02:20 PM
I got the RED SDK some days ago, but I have not had time to play with it.
Just a basic question. Is it possible to just convert/debayer a R3D file/frame with just the REDLINE tool? ...or do I need tons of extra libs?

..and how would this command look like?
>redline -? -?

Mat@imageWork
01-02-2009, 02:13 AM
It's a very good news the r3d support for piranha. Is the plugins avaible to purchase? The multithreading would optimise to work on quad CPU, and/or for next generation of 6 CPU?

Kyle Mallory
01-02-2009, 09:28 AM
Mat,

The plugin is still in development, but is suitable for limited testing. There is still a lot of work to be done.

In testing on my dual processor, dual core system, all 4 cores sit at around 80% utilization. I hope to test on an 8 or even 16-core workstation at some point, but I don't know anyone that has one, so until someone lends me one, or I can find the money to build one, I can't really tell you how much it will optimize-- but in theory, yes it should.

Jason A. Evans
01-04-2009, 04:08 PM
hello im new to Linux. where do i get Piranha?

Kyle Mallory
01-04-2009, 04:26 PM
www.ifx.com

Kyle Mallory
01-11-2009, 02:26 PM
Updated the Plugin status above...

Mat@imageWork
01-12-2009, 11:36 AM
Hi kyle,

Very good news one more time. I just got a phone with Tom. I'll test pirahna in high-end VFX work in the next mounth. Here is the link to our production compagny www.frenchcancan.ca

we have a smoke system here and I will compare the two system. Exiting time...

Mathieu

Vladimir Eugene
01-18-2009, 09:12 AM
Hi Kyle,

I checked out your site, it looks great.

Kyle Mallory
01-18-2009, 01:43 PM
Thanks, Vladimir, but its not my site... Though, I'll be sure to pass along the kudos to the proper individual.

Kyle Mallory
01-28-2009, 11:25 AM
Just wanted to update... Not much news, other than "its working". I have been putting it through a good solid test, as I have been refinishing an old short that was originally cut on FCP via proxies, on Piranha with the original R3D files. Performance is still a concern. There are also some audio issues, particularly with more than 2 channels enabled in the camera, but otherwise, the plugin is performing solidly.

Kyle Mallory
02-03-2009, 03:58 PM
Updated status, and linked the first of the screencasts.

Mat@imageWork
02-06-2009, 12:15 PM
Updated status, and linked the first of the screencasts.

Hi kyle

I'm check some stuff with the SDK on my side. How many CPU are in your system actually? how loaded is their activity? Because having more CPU doesn't mean the sdk will be abble to use them...

Sven Seynaeve
02-06-2009, 12:38 PM
Certainly would like to see it in action. From what I've heard this seems a very promissing tool for finishing red footage, even in stereo which , according to my source seems to outperform some other systems, but I guess we'll have to see some real evidence for this.

Dan Hudgins
02-06-2009, 01:55 PM
Which Linux will it run under? i.e. will it run under Ubuntu?

What does it cost for a licence to run it on a PC to finish 4K with all features?

What does it cost for a 20 computer licence, and will it distribute the work over many PCs for greater speed?

How long does it take to CC a 4K frame load+CC+save time on a single core CPU with a single harddrive?

Kyle Mallory
02-06-2009, 07:36 PM
Hi kyle

I'm check some stuff with the SDK on my side. How many CPU are in your system actually? how loaded is their activity? Because having more CPU doesn't mean the sdk will be abble to use them...

Mat,

My understanding is the SDK is multithread safe, and currently distributes the encoding over multiple threads). That said the plugin will, under certain circumstances kick off multiple threads for decoding complete frames (decoding 3 or 4 frames simultaneously). In my experience with the SDK, on my workstation, a Dual Duo-Core Xeon, the CPU load is evenly distributed amoung all 4 cores. So, in the off chance that the SDK doesn't support multiple CPUs, the plugin itself will, and the SDK has proven very safe under multithreading.

In short, if you have multiple CPUs, the plugin should take full advantage of all of them. Again, my hardware is old, and I'd love to confirm this on a modern 8-core (or higher) machine, but this is all a hobby/non-paying gig for me, so its hard to justify a $10k machine for "testing".

Kyle

Kyle Mallory
02-06-2009, 07:37 PM
Certainly would like to see it in action. From what I've heard this seems a very promissing tool for finishing red footage, even in stereo which , according to my source seems to outperform some other systems, but I guess we'll have to see some real evidence for this.

Sven,

For my own work, the reason why I opted to code the plugin in the first place, it is proving to be a very nice toolset for RED. If you want to give it a try, you should call Tom at IFX, and arrange a demo. You can get his number from the IFX website (www.ifx.com). Aside from the relationship I've developed with them through the use of their software, I have no affiliation with IFX.

Kyle

Kyle Mallory
02-06-2009, 07:42 PM
Which Linux will it run under? i.e. will it run under Ubuntu?

What does it cost for a licence to run it on a PC to finish 4K with all features?

What does it cost for a 20 computer licence, and will it distribute the work over many PCs for greater speed?

How long does it take to CC a 4K frame load+CC+save time on a single core CPU with a single harddrive?

Dan,

It does run on Ubuntu (I run it on Ubuntu, without incident), though IFX recommends Fedora. You will have to call IFX for pricing. I believe it will distribute over multiple PCs, but I've never used it that way personally. CC is done in real-time through GPU acceleration. Of course, it depends on the amount of CC you are doing (how many operations, layers, etc), but it is fair to say that 95% of the time a single frame takes to go through your flow of load+CC+save, is spent actually decoding the R3D frame... Disk speed is negligible (30-40MB/s), but CPU is intense due to the decoding and debayer.

Kyle

Mat@imageWork
02-07-2009, 01:41 PM
Mat,

My understanding is the SDK is multithread safe, and currently distributes the encoding over multiple threads). That said the plugin will, under certain circumstances kick off multiple threads for decoding complete frames (decoding 3 or 4 frames simultaneously). In my experience with the SDK, on my workstation, a Dual Duo-Core Xeon, the CPU load is evenly distributed amoung all 4 cores. So, in the off chance that the SDK doesn't support multiple CPUs, the plugin itself will, and the SDK has proven very safe under multithreading.

In short, if you have multiple CPUs, the plugin should take full advantage of all of them. Again, my hardware is old, and I'd love to confirm this on a modern 8-core (or higher) machine, but this is all a hobby/non-paying gig for me, so its hard to justify a $10k machine for "testing".

Kyle

Ok to our experience with sdk himself on win xp32, recompile to be used in command line, sdk does not scale up to 2 cores. So we develope an mulithread version to act like a render farm and scale up 8 cores. I can't talk sdk under linux, but try to recompile the sdk alone and monitor the cpu activity. if one thread of sdk gives you more than 100% on 2 cores, then I assume linux sdk is different...

so far we have locked our test to scale up to 85% on 8 cores i7. Since the rest of the apz is 100% gpu, we are given a buffer of 15%.

I will glad to look at your linux sdk test, if you have a chance.

Good work BTW, I have maid some test wtih pirahna, it is good and powerfull, but UI design would need the help of a online editor to reshape it, like curve editor. Piranha needs a good user's guide :)

Kyle Mallory
02-07-2009, 08:25 PM
If one thread of sdk gives you more than 100% on 2 cores, then I assume linux sdk is different...

I'll give this a try, but in Linux you will very rarely (under normal circumstances) see a 100% CPU load, in part because of how Linux handles multithreading. Again, in my testing so far, Piranha processing is utilising upwards of 75% of the total CPU time, evenly distributed across all 4 cores, with perhaps 5% of that actually dedicated to Piranha.


so far we have locked our test to scale up to 85% on 8 cores i7. Since the rest of the apz is 100% gpu, we are given a buffer of 15%.

I will glad to look at your linux sdk test, if you have a chance.

Good work BTW, I have maid some test wtih pirahna, it is good and powerfull, but UI design would need the help of a online editor to reshape it, like curve editor. Piranha needs a good user's guide :)

Thanks,

I know Tom has been busy working on revising the manual lately, so hopefully that will address some of your documentation needs.

Kyle

Sven Seynaeve
02-07-2009, 08:38 PM
Thx, Kyle, I'll try to contact them to discuss the options.

best regards

Mat@imageWork
02-08-2009, 08:13 AM
[QUOTE=Kyle Mallory;370965]I'll give this a try, but in Linux you will very rarely (under normal circumstances) see a 100% CPU load, in part because of how Linux handles multithreading. Again, in my testing so far, Piranha processing is utilising upwards of 75% of the total CPU time, evenly distributed across all 4 cores, with perhaps 5% of that actually dedicated to Piranha.

the sdk under Linux is actually given a better job when distributing single thread sdk (if you have coded this way) on CPU. It should be interesting to see how it is working on 8 cores system. I don't know if i7 architecture optimize a virtual 8 cores under linux. Some info can be find when google it. But it should be a good alternative for you to build a cheap 8 cores system on single socket. So far our test under windows with i7 and optimize sdk are very good. I believe we will be abble to get faster than realtime in 2K High playback.

Kyle Mallory
02-09-2009, 01:28 PM
Thanks, Mat. I just spected out a single chip Core i7 machine... It'll set me back about $2k. Know anyone that needs to rent a RED in SLC for a week? Or some short-form/commercials cut? ;-)

Mat@imageWork
02-12-2009, 05:23 AM
Thanks, Mat. I just spected out a single chip Core i7 machine... It'll set me back about $2k. Know anyone that needs to rent a RED in SLC for a week? Or some short-form/commercials cut? ;-)

I will let you know if someone need a shoot there...

Anupam
05-03-2009, 08:16 AM
Any news on ifxant?? the site shows the recent support for RED.

Kyle Mallory
05-03-2009, 09:20 AM
Yes, Ant has full support for RED. My understanding is that Ant is really designed for RED users, whereas Piranha, while it supports RED, also supports film, video, and live ingest workflows. Ant can do these things, but Piranha has a much broader feature set to support them.

I'm not sure when it will be officially released. Tom would have to answer that, but I know its "soon".

el_diablo
05-03-2009, 10:33 AM
I really think IFX needs much more exposure. Like tutorials for the systems, website reviews and such. I had a really hard time to learn anything about the IFX systems. The Swiss reseller suggested to come to them in Switzerland for a system demo, but this is really hard to do with 10s of projects per month with tight deadlines. And local demo seems impossible (i've asked about a eval period).

Not a good way to boost your sales...

Might risk a blind purchase with Ant pricepoint though.

Alberto Caprioglio
11-07-2010, 10:25 AM
Might risk a blind purchase with Ant pricepoint though.

how much is Ant?

Kyle Mallory
11-15-2010, 08:53 AM
You should contact tom (at) ifxsoftware.com for pricing... I honestly couldn't tell you.

Alberto Caprioglio
11-15-2010, 02:12 PM
You should contact tom (at) ifxsoftware.com for pricing... I honestly couldn't tell you.

Hi Kyle, thanks for your reply.
I'd just would like to know roughly how much money we are talking about.
Before calling them and possibly waste their time should the price be completely out of reach or just definitely un-interesting to me.

I have no idea at all of its price.
So, do you know if it is something close to, say, 1000 american dollars?
1000-2000?
2000-5000?
more than 5000?? I think that would be very expensive for a video editing software; but since it is marketed as a professional software for indipendent production, I guess its more like around 1000.
What do you think?
Anyone knows?
Anyone tried it?

Thanks