Optical Flow Explained: The Key to Seamless Image Stitching

3 min read

blogimageoneThe Insta360 Pro ensures a seamless stitch using the science of optical flow.

360-degree cameras face a thorny problem: How do you take visual data captured by multiple lenses, facing multiple directions, and then produce a final image that blends all of this information into one seamless whole?

Add on the fact that the various subjects of these lenses may be moving unpredictably — crossing from the field of view of one lens to another and then back again, hovering about the zone where two lenses overlap, or shifting between the foreground and background of a scene at different rates— and the problem gets a whole lot thornier.

It’s a little like trying to solve a jigsaw puzzle where the image on every piece is changing in real time. And it’s the reason you’ve probably seen shots that turn out like this:

image_stitch_problemToronto’s CN tower has seen better days. 

The Insta360 Pro solves this problem with a cutting-edge stitching technology that’s based on the science of optical flow.

Optical flow is a subject worthy of an army of PhD’s, but the basic idea is simple enough.

Instead of stitching together images according to a one-size-fits-all rule – essentially programming a computer to match dotted line A with dotted line B – the optical flow approach lets a computer keep track of the actual content of an image, down to the level of individual pixels.

Take the Insta360 Pro, with its six independent lenses, as an example. An optical-flow-based stitching algorithm allows for a granular comparison of the six images captured by the Pro’s lenses, so that the images can be stitched together intelligently, according to their actual subjects.

In practice, this means that the Insta360 Pro can avoid taking shots like the one of Toronto’s CN tower above, where a key object has fallen victim to an indiscriminate stitching algorithm. Have another look:

image_stitch_problem

The problem is obvious to any human, but how do you explain it to an algorithm? If the stitching algorithm had been able to notice the stark shift in the arrangement of pixels at the seam where the tower is cut in half – a sudden jump from gray concrete to blue sky – it wouldn’t have made this mistake. By letting a computer actually “see” what’s happening in an image, optical-flow-based stitching avoids this type of problem.

In the case of video capture, the Insta360 Pro’s optical flow stitching algorithm works on an even more sophisticated principle. Not only does it compare the images captured by each of the Pro’s six lenses for any given frame of a video, but it also compares an image captured by a single lens to images captured by that same lens in adjacent frames of the video.

The upshot: Seamless stitching that works with the content of your shoot instead of against it. Compare the two shots below to see how optical flow can save your subjects:

stitch comparisonOptical-flow-based image stitching adjusts a stitch according to the actual content of a shot. 

Optical flow stitching ensures that creators using the Insta360 Pro can transport their viewers into new worlds, worlds that look just like they should – seamless.

15 Comments

  1. Guy Vaillancourt 2017/03/21 at 3:48 pm

    Génial,quand sera-t-elle disponible a l’achat au Canada ?
    Guy du Québec

    Reply
  2. Aaron Nanto 2017/03/21 at 4:07 pm

    So does this also eliminate a minimum distance with the Pro? For example, if I’m standing really close to the camera (say 1-2 feet) and move around the camera, will there be any stitch lines anywhere in the foreground and/or background?

    Reply
    1. Dan 2018/05/28 at 3:12 pm

      A great question! I also want to know this as I plan to use this in confined spaces. Did you ever get an answer? Thanks.

      Reply
  3. Michael 2017/03/21 at 11:53 pm

    Can this be done live?

    Reply
  4. 朱春龍 2017/03/22 at 12:15 am

    請問INSTA360 PRO 它在拍攝時的最近距離(完美縫合)是幾公分
    我目前使用RICOH THETA S拍攝全景照片,製作房屋銷售網頁,它在狹小空間(浴室)也能完美縫合影像
    但是在大場景時影像畫質就差了

    Reply
  5. Joseph Cathey 2017/03/22 at 7:18 pm

    Amazing! Is the optical flow stitching available in-camera only, or is there software that enables it on a desktop as well?

    We’ve been using the Omni, and while its auto-stitch is better than nothing, it still leaves a lot to be desired.

    Reply
  6. Mimosa 2017/04/23 at 4:58 am

    Great post with lots of imnoatrpt stuff.

    Reply
  7. education online courses 2017/04/30 at 7:04 am

    This is really interesting, You are a very skilled blogger. I’ve joined your rss feed and look forward to seeking more of your fantastic post. Also, I’ve shared your website in my social networks!
    http://studypoints.eu

    Reply
  8. job postings websites 2017/05/05 at 10:18 am

    Good response in return of this question with genuine arguments and telling the whole thing regarding that.

    Reply
  9. how can i buy gold bars 2017/05/05 at 4:23 pm

    Hi, i think that i noticed you visited my weblog thus i came to return the desire?.I am attempting to find things to enhance my site!I suppose its ok to use a few of your ideas!!

    Reply
  10. sales jobs 2017/05/05 at 9:58 pm

    I am really grateful to the holder of this web page who has shared this enormous article at at this place.

    Reply
  11. at home jobs 2017/05/06 at 12:31 pm

    Does your site have a contact page? I’m having problems locating it but, I’d like to send you an email. I’ve got some suggestions for your blog you might be interested in hearing. Either way, great blog and I look forward to seeing it grow over time.

    Reply

Leave A Comment

Your email address will not be published.