foolcat / foolmeta - R3D catalogs and metadatas ingest

[Mikael Lubtchansky]

(see post #50 (http://www.reduser.net/forum/showthread.php?72633-foolcat-foolmeta-R3D-catalogs-and-metadatas-ingest&p=1207793&viewfull=1#post1207793) for infos on the latest version 3.0, including support for QT movie clips)


Hello just wanted to let you know about a new tool I just wrote for mac users :



https://vimeo.com/59292121
http://vimeo.com/foolcolor/foolcat



- foolcat lets you create rich HTML and pdf catalog of R3D clips with image thumbnails and extensive metadata / RMD informations (supports RED One, EPIC and Scarlet)

foolcat works with up to 5 clips per catalog in demo mode and requires a 35 USD license to unlock.

A LOT of metadatas are displayed in a compact space in the HTML catalogs (including HDRx values, camera firmware, color curves, alchemy settings, linked audio info etc)...

You can check a demo catalog, here :
http://foolcolor.net/foolcat/foolcat_example/index.html

Download the latest version from the website : http://foolcolor.net/foolcat


companion app foolmeta is being recoded at the moment so is no longer included in foolcat 2.0... but you can use the old version from foolcat 1.26 :
foolcat_1.26 (http://www.foolcolor.net/foolcat_1.26.zip)

- foolmeta lets you add custom user metadatas (Project, Scene, Shot, Take, Notes, Rating, Color Label, Circled take etc) into RMD sidecar files, and is 100% compatible with REDCINE-X metadatas. You can add them in batch to a whole folder and also on a clip by clip basis.

Mikael

[N_Villers]

Fantastic !! Have been waiting since the demise of clipfinder for exactly this solution.

-noel

[David Battistella]

POssibly the most important add on I have seen released.

Excellent. I just bought it. PLease say that you will continue to support it.

David

[Mikael Lubtchansky]

Yes will add any requested feature... as soon as REDCOlOR 3 is released I'll add it etc

I plan to add a simple R3D player when I have the time and once I am enrolled in the R3D SDK program (right now all is done using REDLine)

PS : Just fixed a bug with foolmeta, thanks to David... please redownload a fresh version from the server if you got version 0.95 before this post...

[Michael Mayda]

I tried it. I liked it. I paid for it.
Fine tool, will be a big help in keeping my clips organized. Good to know RC3 will come quickly. Nice work, Mikael.

[Mikael Lubtchansky]

Thanks. I'm writing a small documentation now altho everything is pretty straightforward... hope to post it later today or tomorrow... else it'll wait a few more hours as I'm flying tomorrow evening for a shoot in southern India.

I wrote this app as I was looking for a way of doing catalogs myself with EPIC footage... it took a bit of time to make it right while trying to pack as much info as possible without being too cluttered... The app is smart enough to only show non default metadatas (i.e. in camera look / saturation change will not be displayed if at default 1.0 value) and same for RMD values, if WB or ISO it unchanged from camera setting it won't be displayed... make sure you roll over everywhere in the HTML catalog as it shows lots of info on the small icons, on the 1st thumbnail (non graded clip) or over HDR, timecode, clip name etc

I haven't included all the possible User Metadatas yet (like Owner, Copyright, Edit, Post Notes, filter etc) but this can be added quite easily if needed. The only things I haven't tested yet is Stereo3D clips metadata and anamorphic. If someone can send me a 1 frame R3D + RMD of such files I'd be glad to add whatever is necessary. Once DSMC get a real photo mode I'll add a mode to make catalog of stills with thumbnails... right now it will only display frame number for each still / marker whenever some were tagged in RCX.

Mikael

[Mikael Lubtchansky]

v 1.0 released

fixes a few bugs with specific RMDs and system config... and an issue with the main window being hidden behind others... thanks to Nick Shaw for reporting those !

[Russ Campbell]

v 1.0 released

fixes a few bugs with specific RMDs and system config... and an issue with the main window being hidden behind others... thanks to Nick Shaw for reporting those !

Just installed this version. Doesn't work on my Mac Pro at all on Lion 10.7.2. Ideas?

[Mikael Lubtchansky]

Hi and sorry for the bug...
Just as a test can you do this :
Go into your system preferences
Then into 'langage and text' then under the format tabs...
If you set it to 'United States' and try foolcat, does it work ?

Thanks for your help and sorry for the issue... I'll try to fix it tonight and issue a newer version. I think this bug is related to the way your machine separates the numbers fro the decimals. Thoight I fixed
It but looks like some issue remain !

[Russ Campbell]

Hi Mikael. Works fine now. Is there a way to generate the catalogue with clips in order of time. Often, when shooting on Red One, I bounce back and forth between drives without formatting for an extra safety copy. On Clipfinder, I simply order the clips according to time before generating the catalogue.

[Mikael Lubtchansky]

Hi, Russ, nice to hear it works now, I found the bug and fixing it now...

Yes right now the catalog is only created using file name sort but I'll look and see how I can add the option to have it sorted by shoot time...

[Mikael Lubtchansky]

version 1.02 just released fixes the issue with international (NON US) system settings.
All should be good now !

[Sam Winzar]

This is exactly what I've been waiting for! Great work!

[Mikael Lubtchansky]

version 1.04 will now sort catalog using clip shot date and time by default...
thanks Russell for suggesting this, it makes sense to do it like that anyhow instead of using clip's name :-)

[Vico Martin]

Congrats Mikael, now works like a charm in non US systems. Like Hans and Clipfinder you are in the very best app ever!!

Thanks Men!

[ericyoung]

Hi Mikael

Thanks for making this application! Bought it to support it's development.

Just tried using Foolcat for the first time (v1.04) on 10x128GB cards of mostly 5KWS Redcode 8:1, 75fps varispeed, 25fps timebase Epic footage.
Late 2008 non-unibody 17" MacBook Pro running Snow Leopard 10.6.8

Redline up to date with current "Release" Redcine-X v506.

First time I tried, there was a corrupt clip, so as I was trying to change user metadata clip by clip, it bombed out with a divide by zero error.
Renaming that corrupt clip so it wasn't recognised as a .R3D file allowed the editing of metadata to proceed and the jpg generation to complete.

Unfortunately, while it made all the jpgs fine, it stalled at "create HTML" - I left it running a good twenty minutes, and it didn't complete.
Quitting Foolcat worked, but left the ::: foolcat ::: create HTML progress bar running on top of all windows, and I could not get rid of it except by restarting the computer.

The "Images" folder and graphics were also generated, but no sign of the HTML catalogue.

I had selected the offset beginning and end, but may have had very short takes which were no more than a few seconds long.

I don't see an option to select sort by a particular metadata field.

I do see the potential of this program. However, I miss certain things about Clipfinder.

1) The Excel spreadsheet layout of Clipfinder with editable fields allow me to see an overview of all my metadata ( with selectable columns ) at a glance.
2) I believe I could sort by any metadata field
3) It was easy to add notes to a clip - at the moment with Foolcat, I have to keep pressing a forward button to get to a clip possibly hundreds or more down the list. If we can't have a spreadsheet layout, maybe a thumbnail browser, or file list table to select. Or at least a backward button as well as a forward selection button. And prioritise navigation ahead of generating a thumbnail - would speed things up.
4) I could save a job and use all the recurring personal and production company data as an editable template for a future job.

[Mikael Lubtchansky]

Hi Eric and thanks for your support.

- I'll try to see if I can detect corrupt clips better. Divide by zero means maybe it had a zero duration or framerate that caused an error later on.
- if you force quit the app you can get rid of the progress bar by simply relaunching the app
- no problem using offset on short clips, if it cannot get the offset it will do as much as it can.
- it will by default sort by shot time/date
- No preset yet but all entered meta is available in recent items list... So you can just type the first letters and it will auto complete or select one recent item from the list...

Did you try with a shorter list of clips to see if that work before trying again with very large library again ?
I did tests with 200-300 clips here on a macbook air, took a bit of time but nothing like 20 mins... I'll do more tests with hundreds of clips and let you know...
All your suggestions are valid and I will look into it. I started foolcat with the intention of making simple catalogs and then added the metadatas ingest option. REDCINE-X can also display those metadatas with columns etc... The frontier is fine between making a simple catalog tool and recreating a full REDCINE X / Storm like app...

Will see what I can add in next few version as to make it easier / faster for large libraries. Just received my RED Software Development Kit so give me a bit of time to see how I can start using that instead of REDLine for my future apps / updates

[ericyoung]

Hi Mikael

It's great to have you on here. And appreciate you don't want to recreate other tools.

I haven't had a chance to retry with fewer clips as all the media has gone with the production.
I'll see if I can find some other footage to try it with.

I did try the first day of the shoot with 6x128GB cards and that failed the same way.

It may not be that large a number of clips, as most of our clips were long takes at 75fps. Maybe it's the number of chunks of each R3D clip?

[Mikael Lubtchansky]

No the number of chunks should have no impact

I'll try to see if I can reproduce the issue on a snow leo .8 / REDCine X non pro 506 release combo...

Let me know if you can try with other clips see if you get the same problem but I think there must be some sort of software conflict on your config... Feel free to contact me directly by email so we can work it out when you have the time... I can send you 1 frame R3Ds to test and try other various things

[Mikael Lubtchansky]

New version 1.1 adds support for REDColor3, REDGamma3 and Unsharp Mask values.

[Mikael Lubtchansky]

FYI I just finished rewriting this app from scratch in Xcode (was just a simple applescript before) using the RED SDK.

It is MUCH faster now and should be more stable / reliable as a general app. It also has been updated to properly recognize and list the latest RED formats (1K, 3KHD etc) but basically produces the exact same cool catalogs.

I will release the app in the next few days, as a free upgrade for all previous users.

foolmeta is being rewritten too and will be released a bit later as a separate app with a much better interface to review and annotate clips.

If some of the registered users wants to access the new app before it is released to help testing it please email me directly.

Thanks

[Mikael Lubtchansky]

version 2.0 now released. Using RED SDK instead if REDline to create thumbnails... much much faster ! Coded with Xcode instead of plain applescript. free upgrade.

price is still 19 USD for now.

foolmeta is still being worked on but it will be worth the wait :-)

[Mikael Lubtchansky]

As I had several new users asking about foolmeta, you can still use the old version included in previous foolcat release...

It is quite rudimentary compared to what the new one will be but it might fill the gap in the meantime to batch add project description metadata to a bunch of clips... and this version is free :-)

www.foolcolor.net/foolcat_1.26.zip

[Sean Sweeney]

Wow! I just started re-using this application (tried the earlier versions and they were too slow and unstable for me). These are great. I'm using them to create nice little summaries at the end of every day. Post folks seems to like 'em, it's a nice metadata overview for the day.

Version 2 is great improvement. Well done!

[Glen Powell]

Yes the post guys love these - easy to glance at what's in the folder and look through the meta data. Also helps in archiving footage.

G.

[Mikael Lubtchansky]

New version 2.1 adds support for EPIC Monochrome files !
several minor bug fixes

[Mikael Lubtchansky]

Version 2.1 was just a quick update to add Monochrome support for foolcat but I'm also working on a complete rewrite of foolmeta... it is taking time and I'm still a few weeks away from having a release version but it might be worth the wait !

here's a quick tease showing where it is heading... foolcat will eventually be integrated inside this app.

http://mikasky.free.fr/foolcolor/foolcat/foolmeta_alpha.jpg

[Harry Lipnick]

Any updates on the new build of foolmeta? Looks very interesting.
Thanks!

-Harry

[Mikael Lubtchansky]

It is progressing well... coded some cool features (marker utilities, a new grid clip browsing with mouseover to scrub thru clips (like iMovie) etc)... It is difficult to avoid recreating a mini REDCINE-X but it will have some cool features not in RCX... trying to make a cool swiss army knife R3D utility but it takes time to create and I'm struggling at the moment with optimization to make it play back at decent speed on small machines... not easy but hopefully will make it work...
also need to add support for rocket, HDRx, integrate foolcat etc

will post here when I have some good news or need beta testers :-)

[Phil Holland]

Any word on this coming to PC Mikael?

[Mikael Lubtchansky]

Hello Phil

Sadly not, I'm just a 'spare-time' developper and my skills are limited to the mac environnement... Maybe one day I will be forced to jump to windows and will look into coding on PC but do not hold your breath yet :-(

[Mikael Lubtchansky]

a new freeware and early beta preview of the metadata ingest app

http://vimeo.com/foolcolor/foolcontact
http://vimeo.com/foolcolor/metabox

[Mikael Lubtchansky]

foolcat was just updated to version 2.3 and fixes a major bug when running OS X 10.6 (snow leopard)

Xcode had a compilation bug under Mountain Lion with the 10.6 Deploy SDK that would crash the app when accessing files.
Lion or Mountain Lion users should not be affected... and this 'bug fix' version does not offer new features.

Thanks David N. for reporting back !

[hoylecd]

Mikael, foolcat is incredible, it's exactly what I was looking for as a DIT. The reports your software creates always impress post and make me look good, I can't thank you enough.

Have you considered adapting your software to work with quicktime files? I would love to use your software on Alexa jobs. Not every feature would have to carry over either, just the simple organization and thumbnails foolcat creates would be plenty to start.

Chris Hoyle

[Mikael Lubtchansky]

Hello Chris

Thanks for your feedback.

Yes I had a lot of requests for Alexa / QuickTime version of foolcat and should do something about this. I mostly develop software for my own needs 'as a hobby' and only shoot on RED myself so I need to find some spare time to develop a QT version of my app... Whenever I'll have something working I will post it here or send me an email so I can let you know. But if I open foolcat to Alexa clips I may need to also support Alexa RAW which I have zero clue about etc... So do not hold your breath yet

[Mikael Lubtchansky]

version 2.4 just released.

- should be much faster in the "Analyzing" part while sorting clips by date/time (could take a while on large 200+ clips batch earlier on)
- couple more small bug fixes

new 2.4 demo or unlocked version can be downloaded here :
http://foolcolor.net/foolcat.zip

[Joachim Hoge]

Hi Mikael.
Happy to see updates comming. We are starting on big and long docu/action sport film and I'm so glad I found your software last month. It will make our life so much easier.

[Frank Cueto]

Bump for Mikale and Foolcat. Great piece of software. I would like to fir to evolve into a tool that would allow for easier input of the metadata and support for alexa/qt as well. Would pay extra for it =)

[David Nardini]

Hi Mikael,
Many thanks for a very useful piece of software (foolcat) ... and the excellent support you have provided (large catalogues now possible & adding support for 10.6.x !).
Cheers & keep up the good work.
David

Searchable custom metadata next ... :thumbsup:

[Harry Clark]

Foolcat is an excellent program. Keep up the good work. As others have indicated here, the reports make the whole camera department look more buttoned up.
Alexa/ Quicktime, as well as Sony formats, would really round out the program.
I personally would pay more for each of those versions. I'm not sure how hard it would be for you to implement (not being a programmer myself)
Cheers,
Harry

[Joachim Hoge]

I recommend everyone to jump on this. It really is a very useful app

[David Battistella]

When are you dropping the big new release on us Mikael?

[Mikael Lubtchansky]

sorry I am busy and only can do quick fixes at the moment...

development on the "other apps" needs more free time so has taken the back seat for now...

[Jonas Rejman]

Great app. Thank you Mikael.

[Mikael Lubtchansky]

updated to version 2.71 : foolcat now detects corrupt R3D clips (those clips from 4.x beta reporting as offline in REDCINE-X etc)
(Thanks Jonas for sending me an example corrupt R3D clip)

[ericyoung]

updated to version 2.7 : foolcat now detects corrupt R3D clips (those clips from 4.x beta reporting as offline in REDCINE-X etc)
(Thanks Jonas for sending me an example corrupt R3D clip)

That's a great addition - is there any way in the SDK to mark exactly how many and which frames are corrupt? Or the start and end timecodes of corrupt frames?

[Mikael Lubtchansky]

Sorry Eric, the app will not detect corrupt frames in good clips.
It will only be aware of those "false take", small 16Kb .R3D file created by 4.x beta firmware (when using pre-record mode I think) that contain no data and used to crash the app before.

I did a script a while back to detect corrupt frames in clips and recover most of what was recoverable in the clips (it would decode frame by frame to a tiff sequence and if the rocket card reported an error on decode it would replace the frame with a dummy one and move on to the end of the clip) but it was an "internal" tool to save a friend's project that I was never polished beyond that and thus never released...

[ericyoung]

Ah OK thanks for the clarification Mikael!

[Elliott Balsley]

I just bought the app. It's very cool, thanks for the continued development.
One request / suggestion I have: Reduce the size of JPEGs. According to Google's PageSpeed Insight tool, they could be losslessly reduced by 3-4KB each, or about 27%. Every bit counts when you're on a weak 3G connection.

[Mikael Lubtchansky]

http://mikasky.free.fr/foolcolor/fcat.png

foolcat 3 is almost finished and is ready for beta testing


- full support of quicktime files (including Alexa ProRes with detailed metadata support, canon metadata is still being worked on)

- foolcat 3 is a free upgrade for all early adopters

- QuickTime reported metadata infos includes pixel size, aspect ratio shot date / time, codec, bitrate, audio tracks, timecode, framerate, duration in frame and second... Alexa metadat includes WB, shutter, ISO, Colorspace, cam and media SN, recordFPS, ND filter value, look, audio reel, 3D, lens name, production, director, DOP, location and user infos

- new UI with larger image preview while processing. new application icon and Leonardo DaVinci cat in UI

- individual clip file size now displayed (both for R3D and QT)


It is a beta so some bugs probably remain with some 'untested' QT files flavor. The web page is not yet updated for version 3...
http://foolcolor.net/foolcat_3beta.zip

you can use this forum thread for general feedback or questions...
and contact me by email or PM for bug reports

[Joachim Hoge]

Great news. Looking foreward to test it out. Is footage from the Canon 1DC and 5D supported?

[Mikael Lubtchansky]

I tested it with some Canon 5D files.
1DC I do not know, but if quicktime player can open them there is a good chance foolcat will be able to catalog them... if not please send me an small test file.

I am working on adding all the cool metadatas for Canon (only Alexa clips are supported for now)

quick update :
5DMKii metadata support should be done soon
for 5DMKiii I found sample clips, was able to confirm it contains all the metadata but still looking for a away to extract it properly in Xcode

[John Marchant]

Hi Mikael,

Thought I'd jump onboard... useful tool!

I'm having freezes on old R1 directories when its starts looking at all the proxy files. Can I tell it only to process the .R3D files?

[Mikael Lubtchansky]

foolcat should have avoid those but I overlooked something... now fixed in version 2.96... same link
Thanks for reporting the issue John !

[John Marchant]

Cheers Mikael, I emailed you on the same point so please ignore :) Can't help being a 'beta' kind of person.

[Mikael Lubtchansky]

Anyone else tried the beta with quicktime clips ?
I have to investigate an issue with 10.6.8 but otherwise haven't received much bug reports.
Surprised as it was the most often request feature update... but maybe everyone now is shooting 100% R3D :-)

[Jonas Rejman]

ProRes 4444 works on a 133 clip directory.

PDF and HTML has no logo though.
PDF still crops the Description too much on the right side of the image.

The app is a great!

[Mikael Lubtchansky]

Thanks Jonas for the report. Yeah this logo display bug is still haunting the app, will see about fixing it for good this week if I can !

Mikael

[Mikael Lubtchansky]

beta 2.991

adds anamorphic support for RED clips (creating de-squeezed thumbnails)

same download link :http://foolcolor.net/foolcat_3beta.zip


previous beta :
- adds support for all the Canon 5D MK2, MK3 metadata, enhanced Alexa metadata display
- Now has the option to sort clips by date, by clip name, or keep whatever folder hierarchy sorting you're using
- Lots of bugfixes for QT support and reported display issues...

[Mikael Lubtchansky]

beta 2.993

fixes the 10.6.8 QT incompatibility
and should finally fixes the client logo display issue under all OS version and circumstances !

[Martin Weiss]

Tried to open a folder with about 200 5DMII clips, but when it starts generating the thumbnails, it just locks up after the 1st clip.

http://i.imgur.com/2QarBwX.png

[Mikael Lubtchansky]

Thanks Martin for the report... Just PMed you to help identify the issue so it can be fixed.
Also let me know if you have Perian installed as I found an issue with this EOL prefpane