kyin-place

Intro & Inspirations:

I was really inspired by Philipp Schmitt’s Camera Restrictaand wanted to use a similar method of crowd sourcing data while using geolocations. Therefore, I made a machine that given a latitude longitude coordinate and a picture taken at the location, will crowd source pictures that people have posted online and fuse them to create a single picture of the location.

Process:

First, I wrote a Python script using Temboo/Flikr API to extract all the picture information given a latitude longitude coordinate as input. I was able to get over thousands of images with one coordinate if the location is a popular spot, which was exactly what I wanted. Next I wrote another Python script to download all these images and save it somewhere in my laptop. However, I realized that it wasn’t realistic to use all the images at one location, since, people take such drastically different pictures. I had to figure out a way to pick our the pictures that looked closest to the “picture I am taking”. With Golan’s help, I was able to use the Openframeworks add-on ofxTSNE to create a grid that sorts all the photos by similarities.  Finally I used a 7×7 grid to find the closest 49 pictures in the TSNE grid from a given image (ideally the image that I take), and put these batch of pictures into Autopano.

Results

The results turned out better than I thought. Some were fairly normal, and just looked like a shitty panorama job, and others turned out super interesting. Below are some results with anti-ghost/simple settings.

Conclusions & Future Plans:

Overall, I really enjoyed working on this project and was really impressed with my results. Since I’ve never worked with Autopano, I did not know what to expect. Therefore, it was very rewarding to see the results of this experiment, which turned out to be better than what I expected. Some things that I hope to do to expand this project is to create an actual machine that can do this in real time, and figuring our a more efficient way to filter through the pictures.

One comment

  1. golan

    Comments from the Group Review

    Look at work flickr is already doing – look @ negative space
    Is there a way to make it more ‘panoramic’ – more of the space
    Awesome example of a creative workflow (leveraging multiple tools to make a unique media object) love the aesthetic of the results
    Totally great and amazing that you got this to work ++
    INSANE

    You could make a “speculative design” video showing the (faked) “real-time” version of what it would be like if you were there and had such a “camera” — walk around with the “camera”, etc.

    Some more documentation (in your blog post) of your process, such as the TSNE plots you mention, for example, would be appreciated. Even just some grids of the source images you used would be helpful, so that we could better understand how these composites were derived.

    The result is incredible. This is experimental capture! +1
    What exactly motivated you to use these tourist pictures? Is there a concept behind it? What is the value of using these images that inherently don’t have any?
    I almost find the pictures without the antighost more interesting than the more worked ones ++ (agree but i think it depends on the example)
    Maybe find a more precise line to follow (i.e. places you’ve been, want to go, places with a political meaning, connected by facts/data etc…)

    Very smooth result image, looks like one person captured it.
    Did you try any with photos from places that you have gone to?
    How did you decide which photo was the base?
    Whoa++1

    What if you mixed photos from different times of day? Worth pushing it further — try mixing even more diverse source images.

    Reminds me of the work that the people who developed photogrammetry did; they made models from crowd-sourced photos and found matching points between them.

    TSNE grid is a very clever way to optimize stitching methods. +1
    Do you ever show the ofxTSNE results? You should. ++++a million

    Play with different projections! A lot of the distortion is just coming from this, and may be obscuring interesting stitching artifacts/combinations.

    Oddly hypnotic. A more consistent way of viewing the visuals (i.e. consistent black space around the photo) would produce a more digestible result.

    The results are a HOT MESS, and I mean that in the best possible way. Congratulations!

    Autopano’s anti-ghost methods, for the eiffel tower, seems to stitch and pick just one or two images to show; the ceiling’s are a much better example of this interesting method. ++

    Damn the TSNE preprocessing to make the stitching work is really great.
    Rather than doing a full TSNE, you could just do a nearest neighbour search on one or two photos (using ofxCoverTree, or ofxVpTree, or even just looping through the images and finding the most similar) and then stitch the nearest neighbours, which would let you run your script way quicker.
    Could you do this same process, but instead of choosing a single location choose a single human subject? +(like grab thousands of images of Obama and stitch them together)+1+1
    ^^ or thousands of images of different bathrooms and make it create one bathroom
    What would happen with more than 49 photos?

    I enjoy looking at both the pre-anti-ghost images and the after-anti-ghost. I would be interested to see a gif of the images transforming into the other.

    Check out the following projects:

    Microsoft Photosynth develops 3D models from thousands of tagged tourist photos:
    http://phototour.cs.washington.edu/

    Noah Snavely, Finding paths through the world’s photos:
    http://phototour.cs.washington.edu/findingpaths/

    Blaise Agüera y Arcas of Microsoft, TEDGlobal
    https://www.youtube.com/watch?v=4LxlhoemR3A