Video "scratch" controls?

[RKelln]

Really enjoying my experiments with vimix, thanks.

I was looking today for controls to "scratch" video, smoothly controlling playback speed and/or frame skipping forwards and backwards, etc, and hooking that up to input. I saw I could do play/stop to an input are there other ways to control the playback? (Outside the Player timeline which seems a bit too small and laggy for fine control?) Am I missing something obvious?

[brunoherbelin]

Hi, Thanks for positive feedback and feature suggestion!

If what you ask is to add video playback control to the Input Mapping, then yes this is possible but was not implemented.
Which controls would you suggest?

It would also be possible to add a jog or some sliders to the Touch OSC remote controller. Would you have suggestions / references to a 'usual' control mode?

NB: there is a zoom slider on the right of the Player timeline ; it can help, but I agree that vimix deserves better playback control.

[RKelln]

Yeah a jog wheel is the sort of input I'm looking for. In my experience it's actually really hard to tune the feel and there are often a couple different modes that feel better for different effects/music.

I guess I'd break it down this way:

1. Playback speed fine-control

• slightly alter the speed of playback to better sync to the music
• (I saw some metronome sync options but couldn't figure out how they work)
• usually best to have some ability to do both small increments and large, small nudges versus perceptual speed changes that are designed to match something in the music
  • that could be a modifier for small / large changes or something related to speed of input change

2. Frame jump

• jump forward and back a specified amount, to let you do nice manual looping effects, fast-forward, etc
• jump could be in multiples or number of frames in relation to fps and bpm?
• small frame jumps act as a jog wheel
• probably good to allow seconds or frames (float = seconds?)

Additionally, in general it can be helpful to have some concept of change over time in relation to bpm and portion of the bar/beat. This would apply a change using one button/input that applies over time (with specified easing if you're really fancy). For example, imagine you set a control for a frame jump of -0.5sec over 1 beat (compared to instantaneous jump). You can do that manually with a jog wheel, but can be nice to have that on a single button that you press on the beat and it smoothly goes back 30 frames over the space of the next beat. You can reuse that in a lot of songs and videos.

As a semi-tangential feature, being able to set bookmarks in videos and jumping to the bookmarks on an input (and or jump to nearest / next bookmark) can also be very useful in conjunction with the above. For example, I have a long video that is matched to the music, but the music is being played live so the tempo is going to be variable. I can use playback speed fine controls to compensate, but being able to jump to a bookmark that I know is the start of a bar lets me realign more reliably at changes in the piece.

Long term it might make sense to allow for some sort of arbitrary user specified function to run between input received and ingestion by vimix. Simple functions could just remap, but more complex versions would let people adjust the feel of the controller.

[alexwigforss]

Also interested in this feature. Unfortunately on win so still on GLMixer until now.
I'm thinking of trying to build this in windows. Am I in for a harsh journey?

[brunoherbelin]

Also interested in this feature. Unfortunately on win so still on GLMixer until now. I'm thinking of trying to build this in windows. Am I in for a harsh journey?

Thanks for the motivation for the Windows Port! It would be so great!

This other issue #37 discussed the challenges for Windows and I am confident that the changes would be limited to a few files (mostly systemtoolkit.cpp). I also could contribute, but for that I would need access to a virtual box with the dev environment for Windows ready (compiler, cmake, git, etc.).

[brunoherbelin]

@RKelln Thanks for the suggestions and ideas. Many ideas are totally feasible.

Metronome : agreed, this feature is not documented and not clear. It needs to be better used and I would welcome your suggestions : but first indeed I should give some explanations. I'll let you know when the doc is done.

Bookmarks: yes this has been in my mind for a long time; not difficult per se, but the user interface to add, edit, and jump-to bookmarks needs to be clean and easy. Would something like that correspond to what you have in mind?

[RKelln]

I'd be happy with anything, but here is an alternative mockup:

• show the bookmark line on the opacity/cut UI as well so you can more easily see how they interact
• color doesn't matter but should be distinguishable from the other UI
  • thin 1px outline on bottom flag can help distinguish overlapping bookmarks?
• have only a 1 or 2 pixel line on timeline to no interfere with playhead UI and handle closely spaced bookmarks
• space for named bookmarks potentially in future?

[brunoherbelin]

Sorry for late feedback! Busy times at work...

perfect mockup, totally feasible!
Will work on it and let you know!!!

[brunoherbelin]

Update on the beta branch : direct input action (with key press, tablet input) allows direct seek to position in a video, i.e. equivalent to jump to a bookmark. Example below: the key 'B' is associated to the seek action to time 20:05.00 in the video:

Other actions available include fast-forward and dynamic change of play speed.
This does not replace the work in progress to change the user interface, but at least the mechanisms behind start to be operational...

NB: before this is available in next release, you can compile from source the 'beta' branch (git checkout beta)

[RKelln]

Great, thanks, compiled and tested the beta branch. Couple notes:

Is there a way to map a control to the currently selected video rather than a specific video?
The fast forwarding had bad performance once the value got past 80ms or so (software decoding).

I'll have to figure out how to hook my midi controller up to see how it will work with those controls. It looks like everything is press/no press rather than sending a value? Was thinking I could map to midi to keys but now realizing that knobs would be better for playback speed adjustments but they send a value (0-127).

EDIT: OK looking at the OSC docs now, let me play around.

[brunoherbelin]

Quick answers:

The fast forwarding had bad performance once the value got past 80ms or so (software decoding).

This is unfortunately the hardware limitation of trying to decode faster than the normal speed

I'll have to figure out how to hook my midi controller ....

I also imagined that vimix could directly support MIDI but do not have the hardware to test and didn't look at this further.

So indeed, for now, a fully customized solution is to send OSC messages to vimix, using a MIDI to OSC converter in between. I would welcome feedback on the solution you find (if all open source) so we can add a tuto or an howto in the wiki of vimix :)

[RKelln]

Once I have all the (open source) toolkit figured out would be happy to write up some docs. I've been playing with Chataigne as a central "conductor"/translator but it may be overkill. Also looking at MIDIMonster.

[brunoherbelin]

quick note: I find that MIDIMonster would be an excellent partner tool for vimix !!! Thanks for the tip!

What would be excellent is if we could together make a wiki page to explain the " input mapping " possibilities: describing the vimix interface and basic features, and adding a section on how to translate MIDI to OSC with MIDIMonter (with example config files).

[RKelln]

Sounds good to me, going to be really busy in March but will document any experiments I do and be able to contribute to docs after March.

[RKelln]

I've got MidiMonster up and working with my APC40 MKII, a basic config file looks like:

; MIDI to OSC for vimix

[backend midi]
name = MIDIMonster
detect = on

[backend osc]
;detect = on

[osc vimix]
bind = * 7001
dest = 127.0.0.1 7000
root = /vimix

; Pre-declare the values so the range mapping is correct
/*/play = f 0.0 1.0
/*/pause = f 0.0 1.0
/*/alpha = f 0.0 1.0
/*/tranparency = f 0.0 1.0
/*/loom = f 0.0 1.0
/*/position = ff 0.0 1.0 0.0 1.0
/*/grab = ff 0.0 1.0 0.0 1.0
/*/size = ff 0.0 1.0 0.0 1.0
/*/resize = ff 0.0 1.0 0.0 1.0
/*/angle = f 0.0 1.0
/*/depth = f 0.0 12.0
/*/lock = f 0.0 1.0
/*/brightness = f -1.0 1.0
/*/contrast = f -1.0 1.0
/*/saturation = f -1.0 1.0
/*/hue = f 0.0 1.0
/*/threshold = f 0.0 1.0
/output/fading = f 1.0 0.0
/output/enable = f 0.0 1.0
/output/disable = f 0.0 1.0
/output/fade* = f 0.0 1.0

[midi APC40]
read = APC40 mkII
write = APC40 mkII

[map]

; bottom faders to source alpha
APC40.ch{0..7}.cc7 > vimix./{0..7}/alpha

; master fader => output fading
APC40.ch0.cc14 > vimix./output/fading

; top knobs
; APC40.ch0.cc{48..55} > 

; device control knobs (right side)
; APC40.ch0.cc{16..23} > 

I'm using the flatpak, I'm guessing that doesn't include the beta branch? I can try making my own flatpak maybe? Could you post some docs for the OSC channels for play speed stuff? Thanks!