Example: Processing footage with Visual Cortex System

This topic presents a process for production of video assets with a video synthesizer as a demonstration of passing footage throughout a two Visual Cortex system. This example builds on Example: Developing effects with Visual Cortex System. These examples process external sources to aid in visual identification of concepts. I will be using a full suite of modules along with two external video sources, a glitch processor, and a video mixer. I am not listing the modules used because the example demonstrates process with reduced tool focus.

Once well understood and supported by hardware, reprocessing footage becomes a way of increasing the functionality of your system and an efficient method of building visual novelty.


  • Do I need all of these modules to benefit from this thread?

The motivated user could make use of the process outlined here so long as they can achieve simultaneous playback and capture. The use of a second RGB source should be seen as equivalent to any available signal in the system.

  • What is your sync setup?

I am generating house sync with AJA GEN10 and distributing it to two AJA KiPros. I synchronize my system via VC front panel with either the KiPros, or a Canon XF105 genlocked to KiPro Ref. I do not think my sync is resolved or should be modeled.

Introduction of assets:

  • External footage was captured in studio and by drone.
  • Studio footage auditions two camera treatment of model on prop. Each camera, Canon XF105, outputs to video mixer, Roland V4EX, HDMI input. Transitions, wipes, and fade to black is demonstrated for review.
  • Drone footage comprises afternoon flyover of lake and forest.

  • External footage to be processed is ingested for initial consideration of content. Time lapse contact sheets are generated at a rate that decreases file size for review.
ffmpeg -i drone.mov -vf fps=1/5 -q:v 1 drone%03d.png
montage *.png x4 -geometry 1920x1080 sheet-drone-review.png

  • Find and replace is used to economize preparation of subsequent sheets.

ffmpeg -i studio.mov -vf fps=1/5 -q:v 1 studio%03d.png
montage *.png -geometry 1920x1080 sheet-studio-review.png

  • A high frame rate contact sheet is prepared to appreciate transitions, wipes, and fade to black indicated above and below.
ffmpeg -i studio.mov -vf fps=1/1 -q:v 1 studio%03d.png
montage *.png -geometry 720x486 sheet-studio-1fps.png


1. Gamma Correction and Key Preconditioning

  • Pass through of studio.mov is established on the video synthesizer and a frame for observation is selected.

  • A horizontal ramp from VC ramp section is output to VC Key input, switch set to positive, and used to compare decoded RGB against luma.

  • The luma signal is output to Arch gamma section input, appreciated below as pass through, square, and log.

  • Square and log are each output to Pendulum to complete gamma correction. Diagram, configuration, and range by hand appreciated as 2fps timelapse below.

ffmpeg -i timelapse.mov -vf fps=2/1 -q:v 1 timelapse%03d.png
montage *.png -geometry 720x486 sheet-gamma-correct-2fps.png

  • The gamma correction setting is adjusted to taste, produced below as .gif. (I have gamma-correct.gif but admin please.)

  • Pass through, comparison, and gamma correction follow.

  • Gamma correction serves as preconditioning for keying.

  • The key is output to the buffer and soloed then corrected with threshold and gain adjustment at the keyer.

  • The DVE will serve as a foundation for an initial treatment that will eventually be composited with drone.mov. The footage is audition grade and characterized by testing of parameters and set. With that in mind however, a timelapse will reveal the compatibility of DVE in the test environment.

  • A high frame rate contact sheet is generated with previously referenced commands.

ffmpeg -i dve-test1.mov -vf fps=1/1 -q:v 1 dve-test1%03d.png
montage *.png -geometry 720x486 sheet-dve-test1-1fps.png

2. Compositing Video Layers

  • DVE has well separated values and aids compositing of two video layers.

  • Pass through of drone.mov is achieved on second deck.

  • The instrument is now reconfigured to composite studio.mov and drone.mov together using DVE as key. Signal flow presented below as diagram (Gamma correction sidechain not pictured).

  • FKG3 foreground mode with adjustment of threshold and softness appreciated here in studio.mov audition.

  • This sheet is compared with FKG3 key mode with adjustment of threshold and softness, followed by inversion.

  • FKG3 in key mode, inverted. Observable is recollected and luma from DVE is output to Staircase.

  • Staircase out into VC Composite Input, confirmed by colorization then Mix switch to off.

  • Feedback from VC Expander RGB out to VC Ch. B input.

(,gif resized to conform to forum limitaitons.)

3. Glitch processing

  • The luma signal is used for glitch processing. In this case, a stackable patch cable is used to present luma to the second VC. VC2 Y out to glitch processing. A WET/DRY balance is auditioned.

  • studio.mov is brought to playhead start and I conduct a recording pass utilizing glitch processor as performance controller, appreciated as 1fps timelapse with selects below.


  • The recorded first pass, firstpass.mov, is prepared for reprocessing. The system is reset.

  • Pass through is established and a luma key is generated.

  • Key is adjusted, inverted (out of frame), and BG is filled with texture.

  • Key is used as VC Composite input and confirmed by colorization.

  • drone.mov is output as luma signal to VC Ch. B for auditioning before proceeding to Staircase, Curtain, and returning key to VC Composite input for colorization.

  • The luma output from VC Expander is sent to second deck for digital processing. Memory palace provides tiling and scrolling in warp mode, RGB, luma. The result is keyed as preconditioning for glitch processing. War of the Ants adds texture to key FG as achieved previously with BG fill.

  • A video mixer receives outputs from VC1 and VC2. The keyer on the video mixer is engaged, and now the glitch processor, the t-bar, and the zoom slider provide performance controls for a second pass. Ergonomic placement of video mixer and performance interface modules can be appreciated. Video mixer LCD provides alternative monitoring in same field of view as performance controls.

  • Prior to recording, drone.mov is played back in reverse and has playback speed doubled, juxtaposing not only the direction, but the rate of this asset’s disposition from the first pass. A second pass is captured to bake in transitions between tiled buffer and first pass, appreciated below as 1fps timelapse.


  • The patch is stripped and the premise is established in conclusion and outlined below.

  • An RGB signal is output to RGB channel. A luma signal is keyed and a texture is constrained to its FG. This luma key is used as VC Composite In, and Ch. B receives VC Expander RGB outputs. Ch. B is colorized. VC component out to video mixer and capture.

  • A single color of Ch. A RGB stack is sidechained for processing. The luma key is matured by filtering. Settings are committed in preparation of final pass.

  • Playheads of studio.mov and second-pass.mov are approximated for playback and final pass is captured, available below as 1fps timelapse.

Final Review and Download


  • Functional size of system is increased beyond actual size when processed footage is reprocessed.