A. Create base mesh and apply UV's. Transfer to sculpting program and subdivide and sculpt. Export the sculpted high poly mesh. This export will limit the polygon count that can be used.
Base mesh:
Screengrab of sculpted mesh in Mudbox:
B. Generate position RenderMaps of both the base mesh and the sculpted mesh. To generate this type of map, import the mesh into a blank scene in XSI and apply a RenderMap property to the object. Select a float format and select Object Space. Set the desired resolution on the basic tab. Supersampling may help but I'm not sure. A preset is provided with the addon (RenderMapPosition.Preset). For the base mesh, be sure to set the Subdivision Render Level in the Polygon Mesh tab of the Geometry Approximation PPG to be the same as you expect to use for later rendering.
C. To get a vector displacement map, the position map of the base mesh must be subtracted from the map of the sculpted mesh. The "vectordisplgen" node in this addon can be used to generate either an Object Space or Tangent Space displacement map. If you do not plan to deform the base object or transfer the map to another object, then Object space would be sufficient. Import the base object and set up the RenderTree as shown below:
The Map_lookup_color node is only needed if you are generating a Tangent Space map. In this case, add a Tangent Property to the object using the same UV's which were used for the position maps and point the map lookup to the Tangent CAV. The vectordisplgen PPG has a check box to tell it whether to convert the map to Tangent Space.
Then, apply a RenderMap property to the base object and generate a rendermap of the surface albedo. A preset rendermap is provided.
This is a tangent space displacement map:
This map is viewed in HDRIShop. The black areas are regions where there is no displacement.
Note that for an object space map, all that is needed is a simple subtraction of the base position map from the sculpted position map. This could be done in an external image editing application. Another option is to keep the to position maps seperate and use the RenderTree to do the subtraction at the time of rendering.
Load the base mesh. Insert the vectordispl node in the RenderTree as shown below:
Again, a Tangent property and map lookup are only needed if the displacement map is in Tangent Space. The PPG must be set for either Tangent Space or Object Space vector input. The scale factor may need to be adjusted.
Set the subdivision level in the Geometry Approximation PPG. Set the Displacement options. I usually use fine displacement with view-dependent refinement. With some maps the length will need to be set to a small fraction of a pixel. Min-Max of 0,7 as in XSI documentation, though I haven't tried to optimize this. Render:
