Multi-view FTLV is a thesis in the Field of Information Technology for the Degree of Master of Liberal Arts in Extension Studies.
The team behind this project is, Manish Kumar, Dr. Hanspeter Pfister, Kalyan Sunkavalli and Dr. William Robinson.
The goal of this project was to use Factored Time-lapse Video data of an outdoor building scene under mostly sunny sky, reconstruct the image at any given point of time and project it on a pre-existing 3D model of that building. It was to be done at a good enough frame-rate where projection can be replayed like a movie on the model.
Please consult the included thesis doc (to be included once the doc is approved) for detailed information about various tools, techniques and algorithms used.
Since a time component is involved in this project, capturing static images in the thesis document is not going to explain the project’s result entirely. Hence this project-page is supposed to provide a brief overview of results achieved.
This section will be updated once the document is approved.
One of the main goals of this project was to create a "Viewer" which can read 3D models. It will be used to do projective texture mapping of factorized time-lapse video data. The image above shows a screen shot of this viewer. All the results shown below have been captured in this application. In order to keep maximum focus on the model, avi's have been captured such that only the model is seen. This image just gives a brief overview of the viewer. For details about various functionalities supported by it, please consult the thesis doc.
Though, results have been uploaded on YouTube, the image quality has been modified by YouTube. I would recommend downloading following AVI files from this site and playing them locally on your system. Running them by clicking on the link might not work, so please download them. You might need to install "TechSmith Screen Capture Codec" from http://www.techsmith.com/download/codecs.asp
Result1
Result2
Result3
Result4
Result5
Result6
Result7
Result9
1. First model, with image reconstructed from factored data on GPU and projected back on the model. Three time-lapse videos were captured for this model, capturing two front faces, and a side face. Other faces have static texture maps. The static textures don’t change over time.
2. First model, with original image projected on it. It can be used to compare the results with the previous AVI to compare the factorization algorithm.
3. First model with factored data. But this time, self shadows are casted over static textures to give a better visual indication. Reconstructed images are skipped from self shadow, because they include the shadow data with themselves by definition.
4. Second model - Factored data with shadows over textures. Similar to step number 3, but with a different model and a different data set.
5. Second model – With one static image projected on front face of the house. In this case, as time changes, self-shadow is cast on the static image as well giving much nicer visual feedback even if there is no time-lapse video available. Not using static texture on other faces, since it is much easier to see the effect without them.
6. Second model – Same as step 5, but in this case, using a dithered shadows, for a smoother shadow projection.
7. Second model – What if this model was west facing instead of south facing? Repeating step 5 to recreate this scenario.
8. Second model – How does the day of the year affect the shadow?
shadow casted by sun on 25th Dec.
Shadow casted by sun on 25th July.
9. Power of projection – This avi is to show the power of projective texture mapping. The model used here is from Google’s 3D Warehouse. It has 17,520 vertices and 33,900 polygons/faces. 3 photos of the actual car were taken from 3 sides, and projected back on the car to get this result.