javascriptwebgl

How to replace a pixel in a webGL canvas?


I want to change the color of one pixel for a webGL canvas by first getting the current value and then replace it with a new slightly modified color. For a 2d canvas context this was easy, but I quickly get lost trying to do the same for a webGl canvas context.

Here is my attempt at first reading the pixel color and then drawing it back with changed color. I would prefer just getting an array, change one value and then putting it back instead of drawing, but I found no way to do this either.

var gl = canvas.getContext('webgl');// get the context of Canvas or OffscreenCanvas
// allocate buffer
const out = new Uint8Array(4);
// read RGBA value of pixel
gl.readPixels(98, 122, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, out);
// change red value of pixel
out[0] = 255;
// convert changed array to Float32Array
var float32Data = new Float32Array(out);
// So far everything works but I don't know how to put the changed values back onto the canvas/context
var vbo = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vbo);
gl.bufferData(gl.ARRAY_BUFFER, float32Data, gl.DYNAMIC_DRAW);
gl.vertexAttribPointer(1, 3, gl.FLOAT, false, 0, 0);
gl.drawArrays(gl.POINTS, 0, float32Data.length / 3);

EDIT Added comments to clarify that getting the pixel color value works. I want to know how I can put the changed value back. Do I really have to draw? Can't I simply "edit" the underlaying data buffer/array?

For example, the 2d canvas has getImageData() and putImageData()

EDIT2 I am writing proxies for OffscreenCanvas.convertToBlob and canvas.toDataURL to alter the canvas before convertToBlob or toDataURL returns. For a 2d canvas getImageData and putImage data exist and serves my prurpose perfectly. Are there any equivalent for WebGL, changing a value in an underlaying buffer/array? The data returned by toDataURL and convertToBlob must still be valid objects


Solution

  • You could just draw the webgl canvas to another offscreen 2D canvas and then use the same code-path as you do for the 2D canvas:

    const ctx = new OffscreenCanvas(1,1).getContext('2d');
    export function applyProxy (canvas) {
      ['toDataURL', 'convertToBlob'].forEach(m=>{
        canvas[m] = (...args) => {
          // transfer image
          ctx.canvas.width = canvas.width;
          ctx.canvas.height = canvas.height;
          ctx.drawImage(canvas, 0, 0, canvas.width, canvas.height);
    
          // modify pixel
          const x = 98,y = 122, imageData = ctx.getImageData(x,y,1,1);
          imageData.data[0] = 255;
          ctx.putImageData(imageData,x,y);
    
          // mimic original call
          return ctx.canvas[m](...args);
        }
      });
    }
    

    This solution should be rather efficient while also keeping code-complexity low.

    If you insist on using WebGL you could use the scissor solution given in the answer you already got, its the least amount of code, albeit a bit hacky.

    const x = 98, y = 122, out = new Uint8Array(4);
    gl.readPixels(x, y, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, out);
    out[0] = 255;
    
    // get current state
    const oldScissorRect = gl.getParameter(gl.SCISSOR_BOX);
    const oldClearColor = gl.getParameter(gl.COLOR_CLEAR_VALUE);
    const oldScissorEnabled = gl.isEnabled(gl.SCISSOR_TEST);
    
    // "draw" the pixel
    gl.enable(gl.SCISSOR_TEST);
    gl.scissor(x, y, 1, 1);
    gl.clearColor(out[0],out[1],out[2],out[3]);
    gl.clear(gl.COLOR_BUFFER_BIT);
    
    // restore state
    gl.scissor(...oldScissorRect);
    gl.clearColor(...oldClearColor);
    if (!oldScissorEnabled) gl.disable(gl.SCISSOR_TEST);
    

    Since WebGL is a state machine, to do this properly you have to ensure to restore the state afterwards, for that you have to query it from the driver like I do in the example, which can be slow, or cache it which is error prone. This is also the downfall of the "proper" WebGL solution as it does require setting up a shader and drawing a point thus requiring a lot of state memorization (assuming we need to be host application agnostic) which is why I won't make the effort of writing that out as it's impractical.