An approach to Photogrammetry; The new generation of photo realism

Photogrammetry in videogames is not a new concept, we were using it for years. This tool is a quick way to add realism and detail to the games, after all, we are scanning objects from the real world, and what's more realistic than that? Most of the AAA games are using this technique, some of the big studios create their own models but now is more and more usual the use of libraries. Megascans is the most popular library and now it is integrated in Unreal Engine 5 (Quixel Bridge plugin).

With my past as a photographer and my interest in the new generation of videogames I was very interested in Photogrammetry and its creation from scratch, so I decided to do a small scene to learn more about the workflow and start to use them in my scenes or future work.

This process takes time, there is a lot of cleaning and optimization work, but hopefully with Unreal 5 and the new Nanite system we will no longer worry about the millions of polys of these objects. For this scene in Unreal 5 I didn't optimize these meshes, each object has millions of polys. Nevertheless, if you need to reduce the polys the photogrammetry's software has options to decimate the mesh easily.

I will not go into depth about the whole process since it is not a tutorial, but I will do a brief to show how it works. I started planning the route to scan some objects in the lovely town where I live. We have to avoid strong lights or shadows on the object that we are going to scan, a homogeneous light is ideal because we can have issues if we want to change the lighting in the render or engine later. There are apps where you can see at what time there are shadows or not on the streets. It is really useful to do the planning before going outside, especially in countries like Spain where we got a strong sun and hard shadows. I planned the day and the time to do it but since I'm living in the UK most of the days are cloudy... so almost every day is good to do photogrammetry here! :D

We need at least 100 pictures per object to have enough references, in my case, I took between 180-300 pictures depending on the object, with a 50mm full-frame lense with my Sony A7s II. I didn't use a chart since they are quite expensive and I didn't buy it just for this test, but I neutralized the color in the settings of the camera and later on photoshop too (Colorist experience certificated :D!) I'll buy one in the future, I used them in my previous work and they are really necessary to work on photogrammetry professionally. I highly recommend the X-Rite ColorChecker Passport Photo 2.

I used Agisoft to create the meshes, it is really great software for this task!

The process can take hours and days. The creation of the model in 3D consists mainly of 3 steps, Align the pictures, create the tile cloud and density cloud (a preview where we can remove parts that we don't need, make adjustments, etc) and then build the mesh. In between these processes and after building the mesh there is a lot of work about optimization, decimation and clean up. Each one of these steps takes hours, depending on your PC and the final resolution of the objects (it will vary depending on the number of pictures/detail and camera resolution)

It can be tedious, but after all the process, the first time you see an object from real life converted into a 3D model with all that detail... is amazing! :D. Here you can see some of the final models after clean them up. You can have a look at the whole project here: https://www.franvergara.com/environment-art

The renders below were made with Marmoset. It was a great way to show the final models, it was my first time using it since I always do the final renders in Unreal.

I'll continue exploring the photogrammetry, as Filmmaker, the possibility of mixing real world and videogames sounds like a great opportunity to dig in :)

See you soon!

Fran

PS: First WIP of the scene with a different initial idea