htmlwebglglreadpixelsserver-rendering

readPixels to base64 (WebGL Server side buffer conversion)


I am using headless-gl to run webGL on Node.js, creating an image dynamically on server. Once created, the image is to be stored in database (MongoDB), before user accessing the image again via API.

Below is the part where the image is generated:

var pixels = new Uint8Array(width * height * 4)
gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels)

The pixels are then converted to base64 (since this seems to be the recommended way to be loaded by Image in client HTML).

var base64Image = new Buffer(pixels, 'binary').toString('base64');

However, the string produced by this buffer can't be decoded to produce an image. May be pixels is not 'binary' type? Or should I just save the pixel string in the database and try to redraw the pixel in canvas pixel by pixel in client (I don't think this is the best way) ?


Solution

  • What do you get from gl.readPixels is bitmap data. I use Jimp library to convert the data to different format and then get base64 image string from it.

    NodeJS code

    var Jimp = require("jimp")
    
    //Create context
    var width = 50
    var height = 50
    
    var gl = require('gl')(width, height, { preserveDrawingBuffer: true })
    
    //Clear screen to red
    gl.clearColor(1, 0, 0, 1)
    gl.clear(gl.COLOR_BUFFER_BIT)
    
    // get bitmap data 
    var bitmapData = new Uint8Array(width * height * 4)
    gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, bitmapData)
    
    // use Jimp library to create an image with a specific mime type
    var image = new Jimp(width, height, function (err, image) {
    
        // assign the bitmap as data for the image 
        image.bitmap.data = bitmapData
        // generate base64
        image.getBase64("image/png", function (error, str) {
            // result
            console.log(str)
        })
    })
    

    Test in the browser if it works for you (JS code):

    var base64Img = "" // TODO paste the result from NodeJS
    
    document.body.innerHTML = ""
    document.write('<img src="' + base64Img + '"/>');