With nothing patched in, and all knobs at zero, I’m seeing variation of up to 5 IRE in the outputs of SMX3 channels. That’s probably within QA spec, but I don’t like it.
That’s correct, 5% of nominal is within spec. It’s usually more like 1-2% or less, but the decision to go 2X gain is likely exaggerating the amount of variation.
It will be much less frustrating to leave all of the attenuverter controls centered, and worry about RGB offsets / scale at the RGB encoder module where you have dedicated controls for amplification, attenuation, and offset on each color channel. Dial in your desired color balance after the patch has been set up. If you are setting up a very complex mixing patch (like a staged pipeline), you can insert Proc modules on each subsystem/image generator’s output, so that you can adjust color balance before the downstream sum.
If you are used to parameters in digital systems, 0% always means Zero, because it’s a number. When dealing with analog systems that have 1% to 5% variation range to the controls, it’s best to not think about a number and instead monitor your output by looking at the screen. Watch the screen for “the point at which red is fully saturated” as you turn the knob and stop when you get there, for example. The controls have been designed to have over range where appropriate, so in cases where you get 101% on a control on one module, and 104% on another… the important thing is that the range covers that 100% mark. And that when you’re in the null spot, no signal from the input jack is bleeding thru to the output jack.
So while I get what you are saying, and there is always room to improve (like I said, 1% - 2% would make me happier than 5%), please realize that if these modules had everything precision zeroed out as if they were for the broadcast test signal generator market, they would be a lot more expensive – and without much to show for it, from the artist’s point of view.
Think of your LZX output like raw mixdown from your band’s recording session or logged footage from a video shoot. Before pressing it into a record, or cutting your video, you need to do mastering or color balancing across takes. I presume this is done like any other video project, with color grading and compression tools in the NLE.