Keep in mind the different frequencies used in video and audio. A lot of the video information is above the audible range and thus not necessarily useful for modulating audio without a lot of slew or quantization.
Possibly helpful info: Using audio modules in a video synth system
Probably more interesting/useful to modulate video with audio than the reverse. Or modulate both with the same LFO.