The poor man’s guide to photogrammetry

Photogrammetry Experiments

Since I started fooling around with Agisoft’s Photoscan a few months back and began posting experiments on Instagram, people have been asking me about my workflow.

I’ve been hesitant to post about it, let alone write a tutorial, for a simple reason: I didn’t have a workflow.

I’ve experimented with scanning outside, at home, with and without a turntable, decimating or retopo-ing models, and with various ways to extract maps.

Lately, though, thanks to some expert tips by photogrammetry guru Tristan Bethe at Humanalloy, I’ve begun to settle on a workflow that may be worth writing about.

First, this is an approach that works for small, free-standing objects to be used as 3D assets in scenes–fruit and vegetable, bread, small sculptures, wood bowls, etc. Scanning large objects, trees, ground surfaces, facades or objects without volume, such as leaves, will require a different workflow and, in some instances, much more complex equipment. Same thing if you want to use photogrammetry to make versatile tileable textures (something I’ve also been experimenting with and may write about later).

Second, this is a down-to-earth, shoestring approach that will yield reasonably accurate 3D models but not multi-million-poly monsters accurate to the millimeter. It doesn’t involve anything fancy, such as cross-polarized lights to extract reflectance maps, needed to acquire very high-end results.

In fact, the point of this approach is to generate retopoed models that will not overburden a large scene but which, thanks to extracted displacement and normal maps, will still render convincingly in reasonably tight close-ups.

Let’s start with one of my latest models: A handsome rock picked up in a Berlin park last weekend.

dsc01560

For something like this, I will use a DSLR (a full-frame Sony A7) with a sharp 50mm prime lens, which is literally the only pricey piece of equipment in my setup–well, apart from the software, that is, but if you’re reading this, I assume you’re equipped on that front.

You’ll also need a tripod, a turntable (in my case, a €20 rotating cheese board), a light tent (in my case an €18.99 foldable Ikea box), a cheap ring flash (I got this one for €49.99) and/or a set of four photography lamps (these will set you back €29.99 for a pair).

One difficulty in scanning small objects is to get rid of shadows. In my approach, which consists in scanning the same object twice in two different positions in order to capture all sides, top-down shadows may make Agisoft think it is dealing with two different objects. Also, you don’t want any shadows in the textures you will extract from the model. Getting rid of shadows is tough. I initially tried to do it by positioning two photography lamps on each side of the object and two next to the camera pointing at the object. It works reasonably well. But over time, I’ve found a ring flash to work better. Since the light comes straight from the lens, all shadows are hidden from the camera, resulting in a satisfyingly flat look. The setup is also a lot faster, making for shorter scanning sessions. And while the photography lights generate tons of highlights on reflective objects, making it often necessary to cross-polarize, the ring flash generates just one faint highlight facing the camera. In many cases, the fresnel effect means that this zero-degree-angle reflection is very weak, making it easy to remove in post. Using a turntable instead of rotating the camera around the object also helps even out the lighting.

The flipside of a cheap ring flash is its low strength. But that doesn’t matter since using a tripod means you can set the exposure to be as long as you want. In general, you’ll want to use the smallest lens aperture your lens allows (I always try for 22) in order to expand the DOF and maximize the sharpness of the image. This may mean an exposure of a few seconds per image (lower if your studio has decent ambient light on top of the flash). I normally place my model and turntable in a light tent in order to even out the lighting even more and obtain as neutral a background as possible, but it’s not strictly necessary. If you have a long exposure, use a remote to trigger the camera and minimize camera shake.

Once you’ve photographed your model from all angles, turning the table by roughly 10-degree increments, turn the object on its head and shoot a second batch of images. The resulting two batches of images should cover your object from all possible angles–left, right, top and bottom. Here, DSC01582.jpg marks the start of the second batch.

explorer_2016-10-01_13-21-23

In Photoscan, we start by scanning the first batch of images with the model in an upright position (I’m not going to go into how to use Photoscan as there are many great tuts online about it). The turntable is meant to fool Photoscan into thinking that you shot the object from many different angles even though your camera is actually static and the model is rotating. Sometimes, however, if your background is not neutral enough, Photoscan will pick up details in it and assume, rightly but annoyingly, that the camera was static, resulting in an unspeakable mess of a scarce cloud. To prevent this, you may want to draw a very simple mask around your object on each photo by using the rectangle masking tool just to get rid of these background details. No need to mask the turntable because it rotates with the object (in fact, I’ve found that on tough models, the wood texture on my cheese board gives Photoscan additional reference points).

photoscan_2016-10-01_13-26-46

This first scan is just used to generate a perfect mask for the object, so I’ll use low dense-cloud and mesh settings to deliver a rough, low-poly model quickly.

Before generating the mesh, make sure to clean the dense cloud of any trace of the turntable, as shown below:

photoscan_2016-10-01_13-28-40

photoscan_2016-10-01_13-29-26

Once this is done, import masks for each photo by using the “from model” option, then export these masks to your photo folder.

photoscan_2016-10-01_13-29-44

explorer_2016-10-01_13-30-31

From here, discard the project and repeat the procedure for the second batch of images with the object on its head.

You should now have a perfect mask for each image in both batches. Time to bring all these images into a fresh Photoscan project.

After importing the previously rendered masks for all images–this time using the “from file” option–rotate the photos in the second batch 180 degrees (this is not always needed, but I’ve found it made it more likely that Photoscan would recognize both batches as being from the same object).

photoscan_2016-10-01_13-31-28

Alternatively, and this is a slightly faster workflow, you can also render masks only for the first batch of photos and then use only a quickie hand-drawn rectangle mask on the upside-down photos. This will leave parts of the turntable visible on the second batch of photos but that’s fine as long as it is not visible in the first batch. If the turntable is visible in both batches, however, Photoscan will go crazy.

With some luck (and making sure you click the “constrain by mask” option when aligning), Photoscan will now align and scan the photos and generate one solid model combining top and bottom sides of the object. (My camera “rings” intersect below because I didn’t exactly put the rock on its head in the second batch, but more on its side–something to avoid if possible in order to maximize coverage).

photoscan_2016-10-01_13-35-44

If you used the method where you only render masks from the first batch, make sure to delete any bits of turntable left in the dense cloud before meshing it.

photoscan_2016-10-01_13-56-16

For a reasonably good model, I use a “high” setting for the dense cloud and the highest number of polys available for the resulting mesh. For the rock, it works out at about half-a-million polys.

photoscan_2016-10-01_13-56-56

We’re now half-way through, with a nice, high-poly, triangulated asset.

photoscan_2016-10-01_13-58-42

The next and final step would normally be to calculate a diffuse texture. The problem is that Photoscan will assign the model pretty nasty automated UVs before doing this, making it impossible to edit the map in Photoshop.

The solution–kindly provided by Tristan Bethe–is to import and edit the mesh into ZBrush before re-importing it into Photoscan for texturing.

zbrush64_2016-10-01_14-00-07

ZBrush is perfect for this because not only does it allow you to clean up your scan and remove any annoying artifact, but it also does a great job automatically retopo-ing the mesh and unwrapping sensible UVs. It almost feels as though ZBrush was designed to edit 3D scans.

First, import the mesh from Photoscan into ZBrush, then duplicate it into a second sub-tool and hide the first version.

zbrush64_2016-10-01_14-00-26

I use ZRemesher with a setting of 5 to generate a nice, mid-poly version of the tool with an all-quad topology.

zbrush64_2016-10-01_14-00-58

This is the model I will use in my future scenes.

zbrush64_2016-10-01_14-01-59

Then, cut a few polygroups, which will define the object’s UV islands (two will be fine for our rock, but you may need more if the object has a more complex shape).

zbrush64_2016-10-01_14-02-26

Using the UV-master plugin, you can now unwrap your model automatically. You now have both a clean topology and easily editable UVs.

zbrush64_2016-10-01_14-02-53

For the next step, unhide the high-poly sub-tool and subdiv your low-poly version once (Ctrl+D). By pressing the “project all” button in the subgroup menu, the low-poly model will conform to the high-poly one.

zbrush64_2016-10-01_14-03-31

Repeat these subdiv and project steps until your retopoed mesh has roughly the same number of polys and the same amount of detail as the original mesh.

zbrush64_2016-10-01_14-04-47

To be on the safe side, this is when I normally export both a fully subdivided version of the retopoed mesh and the low-poly version with zero subdiv levels. With your retopoed mesh now dialed back to the lowest subdiv level (the low-poly version), you can bake 4K displacement and normal maps straight in ZBrush that will contain all the details of the high-poly mesh.

zbrush64_2016-10-01_14-05-58

zbrush64_2016-10-01_14-06-28

zbrush64_2016-10-01_14-07-21

Back in your Photoscan project, import the new mesh (whether the low-poly or high-poly version doesn’t matter since they have the same UVs).

photoscan_2016-10-01_14-08-02

You can now calculate the diffuse texture, making sure to use the “keep uv” option to prevent Photoscan from re-unwrapping the model.

photoscan_2016-10-01_14-09-14

photoscan_2016-10-01_14-11-05

photoscan_2016-10-01_14-11-22

This is basically it. You can now export the diffuse and edit it, alongside the normal and displacement maps, in Photoshop (make sure to flip the ZBrush-generated maps vertically as they always come upside down by default) and paint a reflectance or glossy map, using a combination of the displacement and diffuse maps.

photoshop_2016-10-01_14-14-33

All’s left to do is to import the lowpoly mesh into Max, rescale it as needed, and build a shader using your maps.

3dsmax_2016-10-01_14-17-56

Depending on your needs, the renderer you use, or the complexity of your scene, you can either use the normal map to add high-res details to your low-poly mesh, use the height map for displacement or bump, or just work with the high-poly mesh if you need super-detailed close-ups. In all cases, you will have a nice all-quad, animation-friendly, topology and clean, easy-to-edit UVs.

3dsmax_2016-10-01_14-24-10

This rock is slightly translucent, so I’ll use an SSS material. The render below is with normal map only. No displacement.

3dsmax_2016-10-01_14-23-05

Et voilà. With a bit of practice and some trial-and-error, this should take no more than an hour or two per model on average.

I hope you found this useful. As always, let me know below if you have any question and I’ll try not to less too many years pass before responding.

Photogrammetry Experiments

    • fheiland
    • October 4, 2016

    Hey,
    great to read about Photogrammetry. Do you have a Picture of your Self-Made-Studio, with all the lights, tripod, “light tent” etc. Would be a pleasure to see it.
    Kind regards,
    Florian

    • Mirko
    • October 4, 2016

    Hi
    i have really appreciated this overview on your photogrammetry workflow.

    I have a small and cheap setup with a Sony a5000 which is great on very small objects as well, plus a Foldio photographic set, which i have bought just for the purpose of 3d scanning (but yeah, you can build your own with 0 money), and a turntable which is simply a small lazy susan.

    It works pretty well.

    I’d like to suggest also to try using the edgeloop workflow for the retopology in zbrush, to avoid spiral edge flows
    Particularly useful this tutorial about it
    https://www.lynda.com/ZBrush-tutorials/Setting-up-your-model-retopology/374603/383236-4.html

    During the unwrapping process many weird things could go wrong (such as spiral edge flows) in case we are not limiting our intent to baking maps, but we’d like to elaborate a bit more, or would like to keep unwrapping with a manual approach (i.e. uvlayout or others).

    I also use substance painter and xnormal to bake elements like ao, cavity, normals, thickness maps and so on, from the hi res mesh to the lo res.
    The only big problem i have with generating normal maps with different rendering engines is the visible unwrap seams, which seem to be not easy to fix with just flipping channels or changing/hiding seams in the unwrap islands, so as i usually render with Vray i always bake them in the same engine.

    This other technique for unwrapping objects helped me avoiding this problem as well.
    http://polygonspixelsandpaint.tumblr.com/post/19733351349

  1. @fheiland: I’m afraid not. This is a portable set-up which resides on my kitchen table when using it and in a shelf when no.

    @mirko: Thanks a lot for all these great resources. And yes, V-Ray is very forgiving of seams.

    • fheiland
    • October 5, 2016

    @BBB3VIZ
    I just asked myself where to Position those 4 photography lamps, but I guess the Ikea Box is translucent. Thus you put 2 in the front and 1 on each side of the box.

  2. Actually, I tend to put two on each side of the object but slightly to the front of it, and two on each side of the camera, pointing at the object. I found it to be the best way to eliminate shadows. And you don’t want the tent to cast any shadow on the turntable.

  3. Hello Bertrand,
    great tut as always! Thank you! I wish I had this a few month earlier – that would have saved me a lot of frustrating hours … 🙂

    One question: Why do you first scan the model, clean that model, than make the mask files from the model and than scan the whole thing again? Do you get better masks in this way or a better second model? If I use the turntable (covered with a white paper) I always do 1 additional shot without the object (only the plain turntable with the background). Than I use this image to generate the mask files with Photoscan (Tools -> Import -> Import Masks: Method: From Background file name). 95% of all scans this works fine … and 5% the scan looks like exploded … 😉
    Oliver

    • Lionofjudah
    • November 14, 2016

    “..to make versatile tileable textures (something I’ve also been experimenting with and may write about later).”
    That would be very interesting to read.

Leave a Comment