It was established in the past century that 3D information may be recovered from multiple images under the assumption that the observed scene is rigid. This fundamental result is now a 3D reconstruction technology: one may for instance handheld a camcorder around a building and convert the footage to a dense 3D point cloud automatically on a regular computer. The process may also compute the camera's time-varying position, orientation and internal parameters, such as the focal length. But how about non-rigid scenes and objects? Can one similarly film a handheld piece of cloth and later recover 3D from the footage? To make it clear, this is not about using several cameras to observe a non-rigid object, which would boil down to the rigid case, but about using a single camera, possibly moving, providing a single or multiple images, and recovering 3D from these. The constraints involved in non-rigid 3D reconstruction are not only weaker than in rigid 3D reconstruction: they make it a tremendously more involved problem. Research since about the beginning of our century has nonetheless recently led to some important theoretical and practical results for the case of non-deformable structures. In short, non-rigid but non-deformable structures can be uniquely 3D reconstructed from images in many practical cases. I will present general points on 3D computer vision and then focus on the case of non-rigid objects. I will then show how these results can be used to facilitate augmented reality in laparoscopic surgery, providing the surgeon with a live visual guidance in the operating theatre. Some of the next important steps are to integrate models from mechanics and new problem specific constraints.

-