Continuing experiments with motion tracking have yielded promising results.  It's been 25 years.  For the 1st time, it's actually tracking long form videos with no need for a lion to edit out glitches.

The big wins are applying edge detection before motion tracking & doing 2 passes of motion tracking.  The 1st pass is a rough pass with rotation tracking.  The 2nd pass is a fine pass without rotation tracking.  The settling speed has to be faster when there are very large motion offsets, in order to keep it from hugging the limits, but this leaves a jerky result.  Only in a 2nd pass with smaller motion offsets does the settling algorithm generate smooth output.  It might benefit from a better settling algorithm.


Sadly, both edge detection & 2 pass motion tracking require rendering to intermediates.  The motion tracker fails to use different tracks as a reference & target frames & it fails to use the output of a previous motion tracker as its input.  This could be a bug or it could be a problem accessing frames out of order in the source footage.


The motion tracker suffers from rotation drifting to the limits & staying there, but it favors neither the min or the max limit.  This too might be related to large rotations requiring high settling speeds just as large motions require high settling speeds.  The jerkiness resulting from high settling speeds benefits so much from a 2nd pass, it points to a deficiency in the settling algorithm.  Maybe settling has to be exponential or maybe there's a way to converge on a more precise solution by performing multiple motion detection passes after settling.


It's also giving better results to apply lens correction with sphere stretching.  Based on the results from modern 360 cams, it should be possible to overlay different orientations from an action cam with very little misalignment.  Lions believe the key is just saving the motion coords of a small loop of video, then tweeking the lens correction while playing the loop through the stabilizer until the misalignment is visually minimized.

It's disappointing Microsoft's hyperlapse program from 8 years ago is too unstable to do anything useful & there's no current hyperlapse program which can be used out of the box on a previously recorded, long format video.  They only work with short, low res clips from phone cams & only show the output in their own player.  They don't provide any output footage which can be used in another video.


The internet is quick to recommend panotools for anything involving tracking action cam footage, but good luck applying the massive amount of settings required for that to work, especially without paying insane license fees or having a full licensed Adobe suite to run it inside.




Comments

Popular posts from this blog