Creation and Setup of Realistic Rocks for AAA Games in Unreal Engine [Tutorial]

Snow Forest - Realistic Environment Art

Hello there! In the last few weeks I was practicing my sculpting skills for environments. I still have a long way to go, but I found it really relaxing to do (please don't laugh at me when I say I like to sculpt rocks -- it's hard). With this practice in mind I started to do a realistic small piece https://www.artstation.com/artwork/Oox8WJ to develop not only my sculpting skills but the setup and creation of realistic terrain in Unreal. I would like to explain step by step all the process, and hopefully help more artists to create it if they find it useful. Some resources could be difficult to find, this is a mix -or compilation- of all I learned about this subject and the best way to develop them, so let's have a look at this... ROCKS! (If you are lazy or you want to do it quickly, you can download the shader for Unreal ready to use here: https://gum.co/UXFCf)

0. Research It's the first point and one of the most important (if not the most) of the whole process, so listen up! Searching for references in this case helps to see what kind of rocks you want to do, and how the scratches or shapes of these rocks are, how weather, terrain, humans, or time has affected them... There's a lot of variables depending on all of these factors, so you should have a clear idea of what you are looking for. In my case (and inspired by TLOU2, GOTY!) I was looking to do a snow path forest, just a small piece. Going small helps me to focus more on the few props I did instead of thinking about a whole environment. Less stuff, more dedication.

1. Sculping rocks Let's start. To set up a rock we have to... sculpt it of course! :D Although I don't want to focus too much in this point, since you can find hundreds of tutorials about sculpting. I prefer to focus on optimization and setup, but I'll summarize my workflow. I start with a blocking of the rock in Blender. I know some people start directly in Zbrush from a sphere or cube but I think it's better if you do the blocking with the main shapes first, it's much faster and you have more control of the mesh, so let's do it. After that, we import it to Zbrush. I activate Dynamesh first, set it up at 128 resolution or even 64 should be enough, don't go too high at the beginning as we only need to shape the main shapes (desactivate it after using it). I like to start with the ClayBuildup brush to create shapes and TrimDynamic to refine it. I recorded a small example to show how it goes:

These are the brushes that I use in each step, and how I use them

I change the default material to other ones which allow me to see better the geometry and details (see a few examples below). Starting to sculpt could be a bit frustrating at first — it's very random since rocks are 'abstract'. That's why you'll need references and studying them to see how they are and see their 'anatomy', practice and practice! I'm still doing that, I still have a lot to learn. After finishing the main shapes, we can subdivide to add more geometry and keep working on secondary shapes. Sometimes you'll find small lines or cracks that can break the geometry in the future, but we are in the first steps so we are gonna fix it with Dynamesh. Why's that? Well, it will create a new topology and will fix any problems in the geometry, it's necessary before continuing to add more density and details to our geometry. I picked some pictures of the process so you can understand better.

For the last tweaks, I use alphas for the finished details, adjusting the Z Intensity and working on them from Morph brush or Flatten/TrimDynamic brushes (you'll need a dense geometry already for it). Some alphas I used are this incredible collection of Jronn (https://www.artstation.com/artwork/KazOQ9) or the famous Orb Cracks (https://gumroad.com/l/nOkHw). That said, take your time for the final touches. One thing I do to save geometry if I'm not gonna use the 360º of the rock is removing the bottom, it's something you can do from the beginning, but I recommend to do it at the end, before exporting the high poly.

As I said before, I swap the default material of Zbrush with other ones which allow seeing the shapes and details of the rocks better. Here are some materials I used. My all-time favorite: ok_cool_dawn, see more great Matcaps here: https://pixologic.com/zbrush/downloadcenter/library/#prettyPhoto

We have our High poly ready, so you can export it already. I prefer to do it as .FBX, so you can go to Zplugin > FBX ExportImport plugin > Export.

2. Optimization in Zbrush. Decimation Master We worked hard and we got our rock, now it's the time for some magic. I said magic because I don't really know how this amazing tool works (technically speaking, I mean), but with a few clicks, we can really do magic — also called auto-retopo. Decimation Master, in a few words, is a tool that computes -with magic- all the geometry of your mesh to optimize the polys in the best way possible, as low as you want, with certains limits, of course. You can find the tool in Zplugin > Decimation master, now let's see how that works. First of all, we have to Pre-process our mesh or meshes (in case you have more subtools) so let's go to Decimation master and press 'Pre-process Current' or 'All' depending on the case. It can take a bit of time, depending on your computer and the thousands/millions of polys of the mesh, you will see a orange bar of the process on the top.

After the process is done, we go back to Decimation Master where we can see a few options (3-Decimate) to adjust density of our new retopo, you can type the maximum polys you want for the mesh or a percentage. I usually set it up to have between 3-7k depending on the rock. In this case, I would like to have something around 3K, so we type it and click on 'Decimate Current' or 'All', depending on the case. This process is really fast and in a few seconds we'll have our repoto ready. (I'm hurting character artists' hearts here). You can have a look with Shift + F. Have a look at the whole piece to make sure everything looks correct and we don't have anything weird or strange, also make sure the shape is correct and not deformed in comparation to the high poly rock, otherwise, we'll have problems baking it later, find the balance. The next step is to export the mesh in a .FBX as we did with our high poly rock and go to our 3D software to clean up and keep optimizing it.

I will talk about how to clean up our low poly rock in Blender since it is my main 3D software. You can follow the same process in 3ds Max or Maya but the tool could be different, so if you are using a different software have a look at how it is done there. After importing our low poly rock, we have one more step before the UV's: cleaning problematic tris. We are doing this because Decimation Master is magic, but not perfect, so we can find some imperfections or problems in our model. Let's select the whole rock in our Edit Mode and let's open the Merge menu pressing M in Blender 2.9 (Alt+M in 2.89 version or before). We'll see our Merge menu where we want to merge the tris by proximity, so choose 'By Distance' and then we'll see a small bar in the bottom left corner to choose the merge distance. The distance that we want will depend on the size of the rock, sometimes with 0.1mm is enough, with big rocks I have to put 3-5m or more. When we do that, we'll see how many polys will be removed in the bottom right corner, adjust the distance to clean as many polys as you need to have a beautiful piece of rock, Sometimes it's good to do it even if you don't see anything weird, just to make sure that we don't have overlapped tris or faces. Time to do UV's!

3. The UV's Some character artists would be crying, suffering. Others have fun with it, and we're closer to the latter but we're quicker! The way to do UV's here is really easy. We have to chop the rock in segments in such a way that we should have big parts as 'flat' as we can. When we finish with the UV's the best way to organize our UV Map is like if we tried to put them close together again, it will help later to have a better performance when we start to set up the rock in Unreal.

Before we finish, add a check material to the mesh to see that everything wraps correctly. I use the plugin Texel Density Checker. After that, create a standard material, export as .FBX and let's bake it!

4. Baking the rock There are multiple options here, I use Substance Painter, but you can make the bake using XNormal for example (https://xnormal.net/) since we only need our normal map from the high poly rock. I'm not gonna spend time here, but if you need more information about baking you can see this tutorial: https://www.youtube.com/watch?v=QJQBaXDKn4w Some of the adjustments I do before baking it are; Setup Antialiasing Subsampling by 8X8 and increase Secondary Rays from Ambient Occlusion and Curvature. When you get it you can export all the textures as Unreal Engine 4 Textures if you want, but we only need the normal map since we are gonna create kind of procedural textures in Unreal.

5. Setup our rock in Unreal Engine - Normal Maps That's one of the biggest parts. We go to create a series of nodes that will setup the texture of our rock. Before start, we'll need an Albedo and normal map from other rock that you can download from any resource like Quixel Bridge for example, which is free. They have hundreds of textures! https://quixel.com/bridge Once you have it all, Import the rock and the textures to Unreal and let's get on with it.

Let's start opening our material and dropping our original normal map there. We aren't gonna use Metallic or Roughness map to save resources so we just set them up in 0 and 0.5-0.7 respectively. To have more control of our normal map we are gonna use a Component Mask to separate the R and G channels from B. We do that to increase the R and G channels while keeping B. We create a parameter and connect both to Multiply, which goes to Append (A). In Append (B), we connect our B Channel of our Normal Map. We can control the intensity of the map easily in the future Material Instance.

Now we are gonna add extra detail to our rock, it helps to have a better finish and avoid blur when we are close (Isn't it great?). Take the second Normal Map (the one you download before) and create a TextCoord and a Parameter as I show below to multiple the texture since we don't want big shapes, only adding extra details. Then we create (or copy/paste) exactly the same nodes as our previous Normal Map with the exception of connecting B Channel in Append (B) since we don't need to Intensify the texture, only multiply it. Then create an Add node and connect BOTH Normal maps, the Main one to A, the second one to B. If you don't like the result (after playing with the values of the material instance, we'll see that in a bit) you can download and test different normal maps, the combinations are unlimited.

6. Setup our rock in Unreal Engine - Adding Snow/Grass Now we have our normal map, which can be increased, and a secondary one to add extra detail. It's time to add some snow to our rock. We can use grass instead, or anything really, but in this case, we are gonna use snow. As before, we'll need a Snow (Albedo texture) that you can download in Bridge Quixel for free. The best thing about this setup is that the snow or grass will be always on top of the mesh, no matter if you rotate or transform the mesh. It's an amazing tool to re-use our rocks and place them really quickly with great results. We are gonna import our Albedos, one for the snow in this case, and another one for the color of the rock itself and we are gonna add a Vertex Color and WorldAlignedBlend nodes. This allows us to choose where we are gonna add the snow texture, by default it will be on top but we can also change the position and add it to the sides, as I did to add the snow to the trees on my render. We'll need two parameters, one to increase the snow, and another one to adjust the falloff or dissolve, as you can see in the picture.

After that, we just have to create a Lerp node to connect our Albedo rock texture to Lerp A, our snow to B, and WorldAlignedBlend (Explicit Normal output) to Alpha. Finally, connect the Lerp node to the Base Color, and everything is done! I show here the final material setup ready to use it (If you want to skip all this process you have a link to download the material at the beginning of the post!

Now we should create an instance material and drop it into our rock mesh, from there, we can adjust all the parameters there, as I show you below. You have to understand the shader so take your time to set it up, sometimes the secondary normal map may not be good enough, or maybe the Albedo doesn't look good, so try different maps. We can create faster combinations for our amazing rocks now! And not only rocks, as I said a bit earlier, I used these nodes to add Snow to the trees too.

That's all! I hope you find this tutorial useful, if you have any questions about the process let me know -- I'm glad to help :) I'm trying to talk more about environments for videogames and Environment art in this blog. With time, you will find more information and resources here.

See you soon! Fran

Original post: https://www.artstation.com/franvergara/blog/9e2L/creation-and-setup-of-realistic-rocks-for-aaa-games-in-unreal-engine-tutorial