Need help on creating triangle mesh from height map (JavaFX)

I have a program that generates a height map (2D array of integers from 0-255) and builds a 3D view using a Shape3D "Box" object for each 'pixel' with a height proportional to its value in the height map. This creates a boxy-looking terrain that looks cool. My program also creates a corresponding "Color map" to map what color each box in the terrain should be.

I want to be able to also turn this height map into a mesh that can be textured using the color map.

2D Height and color map

Colored triangle mesh created from height map and color map

(These are images I grabbed off google)

If I get it right, you want to build a HeightMapMesh, based on a grid of 2D points {x, y}, with a given height or z value for each point. This value would be directly related to the pixel's color at the same location of a given 2D image.

Getting the vertices is relatively easy: you create the 2D grid, and you get the color using a PixelReader.

Building the mesh is not that easy, but you could just build a regular mesh based on the rectangular 2D image.

There is also another option: given a number of vertices, you could generate a mesh with a Delaunay triangulation.

This is already implemented in the FXyz library: Surface3DMesh.

To use it, just add the dependency to your project:

dependencies {
    implementation "org.fxyz3d:fxyz3d:0.5.0"
}

The following application will do a rough approximation at the HeighMapMesh you are looking for.

It uses the image you have posted to create List<Point3D> data based on a PixelReader every 5 pixels on x and y, with just a small sample of the colors of that image.

With this list, two surfaces are created, one will be filled and rendered with a texture map based on the height of each vertex, using the same color list. The other one will be used as a wireframe to be rendered on top.

public class HeighMapMeshTest extends Application {

    private static final int PIXEL_SIZE = 5;

    private static final List<Color> COLOR_LIST = Arrays.asList(Color.web("#3b6eca"),
            Color.web("#d7d588"), Color.web("#60a318"), Color.web("#457517"), Color.web("#467610"),
            Color.web("#654f44"), Color.web("#56453d"), Color.web("#fdfefc"), Color.web("#ffffff"));

    private final Rotate rotateX = new Rotate(-10, Rotate.X_AXIS);
    private final Rotate rotateY = new Rotate(5, Rotate.Y_AXIS);

    private double mousePosX;
    private double mousePosY;
    private double mouseOldX;
    private double mouseOldY;

    @Override
    public void start(Stage primaryStage) {
        Group sceneRoot = new Group();

        PerspectiveCamera camera = new PerspectiveCamera(true);
        camera.setNearClip(0.1);
        camera.setFarClip(10000.0);
        camera.getTransforms().addAll (rotateX, rotateY, new Translate(0, 0, -800));

        Scene scene = new Scene(sceneRoot, 1000, 600, true, SceneAntialiasing.BALANCED);
        scene.setCamera(camera);

        List<Point3D> data = processImage();

        Surface3DMesh heightMapMesh = new Surface3DMesh(data);
        heightMapMesh.setDrawMode(DrawMode.FILL);
        heightMapMesh.setTextureModeVertices3D(new Palette.ListColorPalette(COLOR_LIST), p -> -p.y);

        Surface3DMesh wireframe = new Surface3DMesh(data);
        wireframe.setTextureModeNone(Color.BLACK);

        Group mapGroup = new Group(heightMapMesh, wireframe);
        mapGroup.getTransforms().add(new Translate(-500, 100, 0));
        sceneRoot.getChildren().addAll(mapGroup, new AmbientLight());

        scene.setOnMousePressed(event -> {
            mousePosX = event.getSceneX();
            mousePosY = event.getSceneY();
        });

        scene.setOnMouseDragged(event -> {
            mousePosX = event.getSceneX();
            mousePosY = event.getSceneY();
            rotateX.setAngle(rotateX.getAngle() - (mousePosY - mouseOldY));
            rotateY.setAngle(rotateY.getAngle() + (mousePosX - mouseOldX));
            mouseOldX = mousePosX;
            mouseOldY = mousePosY;
        });

        primaryStage.setTitle("F(X)yz - HeightMapMesh");
        primaryStage.setScene(scene);
        primaryStage.show();

    }

    private List<Point3D> processImage() {
        Image image = new Image(VoxelTest.class.getResourceAsStream("/8rF9BXu.png"));
        PixelReader pixelReader = image.getPixelReader();
        int width = (int) image.getWidth();
        int height = (int) image.getHeight();

        List<Point3D> data = new ArrayList<>();
        for (int y = 0; y < height - PIXEL_SIZE / 2; y += PIXEL_SIZE){
            for (int x = 0; x < width - PIXEL_SIZE / 2; x += PIXEL_SIZE){
                Color color = pixelReader.getColor(x + PIXEL_SIZE / 2, y + PIXEL_SIZE / 2);
                float h = Math.max(COLOR_LIST.indexOf(color) * 10, 0);
                data.add(new Point3D((float) x, -h, (float) (height - y)));
            }
        }
        return data;
    }

    public static void main(String[] args) {
        launch(args);
    }

}

As a result:

Of course, this can be improved in many different ways.

EDIT

Once the 3D mesh has been created, it can be exported to an .OBJ file, including the texture applied.

FXyz already includes OBJWriter for this purpose.

This code:

OBJWriter writer = new OBJWriter((TriangleMesh) heightMapMesh.getMesh(), "mapHeight");
writer.setTextureColors(9);
writer.exportMesh();

will generate mapHeight.obj and mapHeight.mtl, where a diffuse image named palette_9.png is used.

However, this palette image doesn't use the custom palette we have defined.

In order to export the custom colorPalette, we need to create a Palette, and save it to disk:

OBJWriter writer = new OBJWriter((TriangleMesh) heightMapMesh.getMesh(), "mapHeight");
writer.setTextureColors(9);
Palette.ListColorPalette colorPalette = 
        new Palette.ListColorPalette(COLOR_LIST);
Palette palette = new Palette(9, colorPalette);
palette.createPalette(true);
writer.exportMesh();

Verify that the palette file is a 3x3 image with the colors from COLOR_LIST.

Now you can open the obj file with 3DViewer to check that it was exported correctly.