Pixelmania

We haven’t posted any new releases for a while, so we feel it’s time to give you an update of what we’re currently up to and what’s to come. But first a little bit of a background by looking back at the year so far. The year started with the release of our third Nuke plugin NNCleanup. After getting v1.0 out in February, we quickly followed up with a couple of maintenance releases. When we felt NNCleanup was in a stable state we circled back to our most popular plugin NNFlowVector.

We adapted a modern and new transformer based optical flow solution to the plugin and trained the network from scratch using the same dataset as the already released solution. After a few months of work we released NNFlowVector v2.1 with the new “AA” variant. This has been welcomed by lots of users across the globe, so we are very happy with the addition.

After the release of NNFlowVector, we felt it was time to circle back to our first plugin NNSuperResolution. This is where we are spending all development resources currently. The most important thing we are doing to it is swapping out the rather old and simple optical flow solution that was baked into the sequence mode upscale solution. We are replacing it with the very successful model from our NNFlowVector plugin. With this new and very performant optical flow generator the upscale solution is able to much more reliably warp in the previous frame to the current and hence have more info to work with. The result is overall a sharper and more stable image with less artifacts. It does mean retraining the whole solution from scratch though, so it’s taking quite a lot of time. All variants need retraining; Alexa 2x, Alexa 4x, CG 2x and CG 4x and they are each a two step process taking between a week or two in processing time. We are also looking into the possibility of a CG/RGBA solution for stills, basically so you can upscale textures with an alpha channel. This solution is not working satisfactory yet though, so we are not making any promises about it just yet. The aim is to get NNSuperResolution v4.0 released in the end of the year.

After the new NNSuperResolution is released, it’s time to circle back to NNCleanup again. The big thing to attack then is a sequence mode, i.e. being able to cleanup/paint away objects in moving material. This work has already started, but it’s very complex so it will take quite a lot of development time. The aim and hope is to get NNCleanup v2.0 released, with a working sequence mode, some time during 2024.

Cheers,
David

We have officially released v2.1 of NNFlowVector. The new version features render license support, Nuke Indie support and a new variant called “AA” that is based on a new and modern transformer model. You can go ahead and download it now!

The render license support means more flexibility when licensing our plugins. You can for example buy a couple of GUI licenses for your artists to use interactively when working, and then have a bunch of render licenses for the farm to batch process with (without them interfering with the GUI licenses). If you are a bit technically creative, you can also make your farm use the GUI licenses durings off hours to up the license count for overnight batch rendering on the farm.

Nuke Indie support opens up the use of NNFlowVector for more Nuke users. All of our plugins are now supporting Nuke Indie, and are listed as such on Foundry’s 3rd party listing of plugins for Nuke.

The new “AA” variant is an even more complex neural network for solving optical flow / motion vectors for image sequences. It can handle some complicated situations better than the already existing “A-H” variants. Due to its increased complexity it is also using more VRAM on your GPU than the other variants. To be able to fit it on your GPU, you might need to lower the “max size” parameter. We have also introduced a new parameter called “separate inferences” that will solve the forward and backward pass after each other instead of in parallel, which also makes the plugin use a bit less memory. Worth knowing is that the new variant is not always better, so it’s still recommended to try out and compare a couple of different variants for your own specific image material.

Take care and happy compositing!
Cheers, David

We’re now officially featured, with all of our plugins, on Foundry’s 3rd party listing of plugins for Nuke.

We are pleased to announce that we have released both NNSuperResolution (v3.3.0) and NNCleanup (v1.3.0) with dedicated render license support. This means more flexibility for you to choose how to license our plugins. You can for example buy a couple of GUI licenses for your artists to use interactively when working, and then have a bunch of render licenses for the farm to batch process with (without them interfering with the GUI licenses). If you are a bit technically creative, you can also make your farm use the GUI licenses durings off hours to up the license count for overnight batch rendering on the farm. Adding render license support was in response to customers asking for this functionality, and we do 100% agree it’s a good thing! If you already got a node locked license, or if you’re a studio with a site license, there is no need to worry. You don’t have to change anything, and things will keep working as it is currently (i.e. the nodes will continue rendering using the GUI licenses in those cases).
NNFlowVector will also get the addition of render license support in the next point release, which will probably be available in a few weeks from now.

The render licenses are available for purchase in the Shop already, and are priced at $59 USD/year.

NNCleanup (v1.3.0) has also got support for Nuke Indie with this release!
NNFlowVector will follow along with Nuke Indie support as well in the next point release.

Since our last blog post in February, we have also released NNFlowVector v2.0 as a sharp public release (i.e. no beta version anymore). This means that you can now go ahead and download and install it, and use the matte input support on both Linux and Windows.

Until next time!
Cheers, David

This new version features some new and important controls for what area to process when doing the cleanup/inpainting. You have the option to either process “Full frame”, “Specified region” or “Matte input’s bbox” (the old behaviour of v1.0.0 was always “Full frame”). Because the area that needs processing is usually just a sub section of the image, having these options makes it possible to work on really large images, 4K plates and even 16K HDRIs for example. We’ve also added a “process_scale” knob that makes it possible to work on really large areas by internally downscaling them before processing (and then later upscaling them again). All this makes the memory footprint way smaller on the GPU and hence possible to still work GPU accelerated.

We have added a couple of HDRI examples on the product page, including downloadable EXRs of before and after.

Cheers,
David

We’re excited to announce our third product NNCleanup, a Nuke plugin for quick cleanup / inpainting tasks. Read more about the plugin, and have a look at a few more examples, at the product page. If you are keen to try it out yourself in Nuke, please request a free trial license. We already got ideas for improvements, but until then let us know what you think using this form, we are keen to hear your input!

Cheers,
David

We have just released a public beta version of v2.0.0 of NNFlowVector. This version features a matte input, so you can make the plugin ignore a selected area. To be more precise, it’s not really ignoring the selected area but rather treating it with inpainting and some machine learning/artificial intelligence to make it appear as an approximation of what it should have looked like if the objects selected weren’t there during filming. This makes it possible, for example, to create motion vectors of the background wall even if a character is passing by in front of it. You can then use the vectors to track in footage/image patches onto the wall. You still got to roto back the character of course, but the tracking is solved.

We have decided to make the v2.0.0 release available as a beta version because we are really keen for you to get your hands on the matte input feature. This has been the single most asked for feature by you, so we are very excited to deliver on that! The choice to release it as a beta version first is to get it into your hands earlier, instead of postponing the release a couple of months due to extra testing. We hope that the plugin is already stable, but if you are experiencing bugs or crashes, please drop us a mail at [email protected] where you are explaining what is going on. Thanks for the help!

Cheers,
David

We have just released builds for both NNSuperResolution and NNFlowVector for Nuke 14.0. What’s good to know is that Foundry has updated the bundled CUDA version in Nuke to v11.1.1, and the cuDNN version to v8.4.1. We have matched the builds of our plugins to that so there are no compatibility problems. You will basically get native compatibility of all supported GPUs up to compute capability 8.6, i.e. Ampere type of cards (for example RTX3080 and RTX3090). If you are lucky enough to own a brand new RTX4080 card or similar, you will have to rely on the JIT compilation of the kernels. The plugins will work, but you will need to wait for that kernel compile the first time around (there is more info about this in our documentation PDF). Enjoy!

We are also pleased to announce that we are very close to releasing NNFlowVector with matte support! This will be released as v2.0 in the beginning of the new year. We are very excited about this since this is the most common feature request we get. If you are very keen to test this out and can’t wait, we are interested in having beta testers. Please use the normal contact form, and please let us know what build you are using (platform, Nuke version, CUDA version). We will then send you an email with a special download link so you can get up and running.

Hope you like our Christmas presents. 🙂
Merry Christmas and Happy New Year!
Cheers, David

We have successfully migrated all of our files available for downloading to Amazon’s AWS S3 cloud system. We received your feedback about our downloads being painfully slow (for some of you it even took days to download the latest builds of our plugins). We agree that this was not acceptable, and have now solved it by upgrading to a much more solid solution. We hope that this will provide nice download speeds going forward, no matter where in the world you are located.

Cheers,
David

We have just released a new version of NNFlowVector, v1.5.1. It’s a patch/bug fix release with the following release notes:

• Patch release fixing a streaking error that occurred with the last processing patch (furthest to the bottom-right area of the processed image), in some resolutions (resolutions that needed padding to become dividable by 8).
• Improved the blending of the seams between processing patches. The problem was not always visible, but became apparent in some specific combinations of maxsize, overlap and padding values.

We have noticed that the NNFlowVector Utility nodes “MotionVector_DistortFrame” and “MotionVector_FrameBlend” don’t work in Nuke13.1 and Nuke13.2. They do however work in Nuke13.0 and earlier versions. We investigated this and found the reason to be a bug in Nuke in the way it handles motion vector channels in the IDistort node when they are time shifted using a TimeOffset node. If you are interested, here is the bug ticket (ID 518631) at Foundry’s site: https://support.foundry.com/hc/en-us/articles/7496106165010-ID-518631-The-Viewer-outputs-a-grey-image-when-there-is-an-IDistort-downstream-of-a-time-node

We have recently relocated our website to a new hosting service. It’s now much faster and more reliable. We are currently looking into moving our file hosting service as well, to get all downloads working better (we know that they have been painfully slow as times, thanks for letting us know!)

Due to client requests, we have added a new license type for our plugins called a “Global license”. This type of license is tailored for large companies that operate world wide, i.e. on a global scale, with operations present in a lot of different geographical locations around the globe. You will find the option in the Shop next to the other types (“node locked”, “floating” and “site license”).

Have a nice weekend!
Cheers,
David