What is photogrammetry?
Anyone who is interested in the possibilities of 3D model creation will sooner or later stumble across the term photogrammetry. This is a technique that allows to generate a digital 3D model out of photos of a real object.
Source: Screenshot of a photogrammetry model, monument at Schlossberg/Graz, created by me
The term photogrammetry is widely spread and includes e.g. surveying methods from geodesy, but we want to deal here with the area of application that makes it possible for us to calculate polygonal 3D models from photos. This technique makes it possible to create realistic 3D images of real objects and to integrate them into our Games/AR/VR apps or visualisations without the need for an experienced 3D modeler to create the objects “by hand” in a 3D program. But don’t panic dear 3D artists, nobody has to fear for his or her job, because like everything in life, this technique has both advantages and disadvantages. But since some of the advantages are considerable, we want to take a closer look at frequently used photogrammetry solutions!
Software Solutions
There is already a not so small selection of common software solutions for photogrammetry, we’ve listed some of the most commonly used solutions here. Some of them are available for free, but most of them are not free of charge but offer trial versions in case you want to try them out or test their performance before you decide on which one to get.
CapturingReality Reality Capture (trial available)
Agisoft Photoscan/Metashape (trial available)
Zephyr3D (free version available)
VisualSFM (free)
MeshRoom (open source)
Source: https://www.agisoft.com/index.php?id=38
Best Practices
Basically, photogrammetry works as follows: You load the photos from the object into the software, these are automatically adjusted by feature matching between the photos and a point cloud of the object is generated. This point cloud can then be refined before creating a 3D model (consisting of triangles/polygons) and the corresponding image texture. You can control some parameters in the software, but most of the time the quality of the model depends on the photos taken, so I’ll go a little bit deeper into how to take these photos ideally:
As the first object to get familiar with photogrammetry I would recommend a large stone or tree stump, in no case an object that is mirror-like, has metallic or even high gloss surfaces. These are not particularly suitable for 3D model reconstruction, but there are ways to process these objects by photogrammetry, e.g. by using special spray paints that are easily washable but provide a matte surface. But this is only for advanced applications and we won’t go into it in detail now. The ideal case is when the object is a little isolated, so that you can photograph it well from all sides.
You can always photograph outdoors or in a studio. In terms of exposure, the same applies to both scenarios: No direct bright light that casts hard shadows, ideal is a diffuse lighting that does not allow any conclusions about the position of the light source. This is important so that the objects can react dynamically to the existing lighting in other applications, which is of course not possible with fixed shadows in the model. Practically this does mean no taking pictures outside in direct sunlight on a sunny day, ideal weather is “Overcast”: cloudy but still quite bright but no direct sunlight. If you can’t wait for such a weather, it’s best to look for an object that is completely in the shade, this should be fine as long as there are no visible hard shadows from the sun.
Then when photographing with the camera (SLR recommended, but smartphone also works surprisingly well), be sure to set it to Pro mode, so that all automatic settings are deactivated. No matter how you set it in the beginning, if you change the settings in the middle of the photo session (e.g. because the autofocus adjusts itself or the aperture, or …) then firstly the photogrammetry softwares algorithms can get confused and secondly you can get artefacts like brightness gradients into the texture.
The ideal settings would be the following: Make sure that the brightness and contrast fit well, ISO as small as possible so that you have as little sensor noise as possible (important for good feature point recognition later) and focus set so that the object and a little environment in front of/behind it is razor sharp. Whether the background is blurred doesn’t make much difference in my experience, often it helps even when everything is in focus because the algorithm can orient itself to the farther background and the camera alignment improves in the software.
When taking photos, go around systematically yourself and take hemispherical (or spherical if you also take photos of the underside) photos of it. So the camera perspectives should form a hemisphere around the object, the photos should always overlap by about one third of the image, this is important for alignment. If you don’t want to photograph a single object but something huge like a house facade and then reconstruct it in a grid pattern (see picture below).
It is important to change the camera positions and not only the viewing angle from always the same position as shown in the following graphic:
Optimization
Due to the high accuracy of the reconstruction, the resulting 3D models are of very high resolution and it is not uncommon for them to have several million polygons. That such models are a big performance problem, especially on mobile devices like smartphones and tablets, is easy to understand. Therefore a subsequent optimization of the models is usually unavoidable, in order to explain how this works best I refer to another blog post of mine: 3D model optimization for mobile devices.
Conclusion
Photogrammetry is a very powerful new tool for 3D model reconstruction that has become indispensable in modern content creation pipelines. In combination with expert users you can increase the quality of the created 3D models as well as the efficiency of the model creation. If you have further questions on this topic, please do not hesitate to contact us!