June 02, 2010

Projection/Displacement workarounds

If you're serious about high-end 3D content generation, you'll eventually run into "displacement map" generation (that thing that makes a low poly model become super-detailed at render time). As Bill Gates probably once said, "with huge power comes huge bugs", and so it's almost impossible to avoid the displacement maps associated limitations and inconsistencies across applications. Here at FluidPlay Studios, we've been suffering a lot with displacement generation in zBrush as of 3.5. To project the high-frequency details of some 3rd party models into our re-topo'd low poly meshes we've had to resort to multiple gymnastics, here's a quick rundown:

- Divide low poly cage only up to level 3 (keep original base stored as morph target in level 1, ofc)
- Use project all. Limiting yourself to level 3 will mean project all won't completely ruin your day. Just partially.
- Send the level 3 model to Topogun to re-snap the tricky edges to where they should be relative to the high poly model.
- Bring level 3 mesh back again into zBrush, import it over level 3 ofc
- Divide a couple more times - we avoid going over 6 to keep fixes manageable, also detailed enough if your cage is well topo'ed. If your edges are really tricky and you need to preserve them, try dividing to higher levels with smoothing off, or maybe one divide with smooth off and the next one or two with smoothing on. Your mileage will vary.
- Super-important step - MASK THE EDGES. Edges don't project automatically well, period. Go up and down sub-d levels as needed to make this easier.
- Soften the masked edges and run another project all (this time you should be on level 5-6)
- Turn on that *blessed* culling auto-mask feature in the brush menu, that'll help tons with the project brush in edges
- Invert the masked selection and start using the project brush to bring in some edge detail. I recommend not being too picky here, it's generally not worthy, if you did a good job in the topogun phase it should be enough.

From there on it you should get a clear map by turning off adaptive (it's broken in 3.5) and using 32-bits. Remember to restore your morph target in level 0 before generating the map, and turn on Flip V if you're sending it to Mudbox.. or anywhere else for the matter. Cra-Z-brush and its annoying non-standardnesses.

Anyways, best bet by now, if you've also got Mudbox, is probably using it to generate displacement between a mid-poly mesh and a high poly one. A little trick if you go down that route: use decimate in zBrush first, to pull the poly count to slightly under 1 million. Otherwise Mud will fall down to its knees and probably crash or be to slow to be usable. Also don't select the objects using the object manager, just "add all" for each item (low/hi) in the map extraction window and remove the one you don't want. Life saver. Just remember to check your UV subd algorithm vs the one that your render app uses (for instance, Renderman uses smooth UVs). Also notice that Mud doesn't have any "project" brush so you can fine tune its projections, so you might need to go back to zBrush to manually fix some issues (always a tricky trip when it involves displacement), or just use photoshop to fix things by hand if they're minimal.

Oh, for the record, Vector Displacement Maps (VDMs) are already a reality in Mudbox 2011, and can be easily rendered in Maya/Mental Ray using Wayne Robson's plugin. Differently from standard displacement maps though, don't expect to generate VDMs in Mud for anything that comes from the outside - which's quite narrowing for most pipelines.

Well, that's it for now folks. Good luck!


Post a Comment