androidandroid-jetpack-composeandroid-canvas

Jetpack Compose - best way to handle pixel-exact composables with pointer input


Jetpack Compose seems specialized for UI that is positioned and sized automatically, relative to the rest of the UI, without needing to know its exact size. This is great in many cases, but I have a few applications where I need a composable to draw at exact pixel coordinates and sizes. I also need to to track pointer input on the composable, as a percentage of the composable's size.

enter image description here

The specific case I'm working on now is displaying a piano keyboard, shown in the picture, where each key needs to be sized and placed at specific positions relative to the keyboard as a whole. Also, I need to track the pointers within the keyboard and calculate which keys are pressed. I'm currently implementing it using a Canvas composable. It's helpful that the canvas provides you with its size, but I can't use that size outside the scope of the Canvas for use in tracking the pointer input. This is my plan as of now.

    var width by remember { mutableStateOf(0) }
    var height by remember { mutableStateOf(0) }

    Canvas(modifier = modifier
        .background(Color.Black)
        .pointerInput(width, height) {
            awaitPointerEventScope {
                 /* track pointers and calculate which
                 keys are pressed using the values of
                 width and height */
            }
        }.onSizeChanged {
            width = it.width
            height = it.height
        }
    ) {
        /* draw the piano keys, using the values
        of size.width and size.height to size and
        position them */
    }

My questions are:

  1. The use of the .onSizeChanged modifier to keep track of the size seems janky. Is there a better way? I'm also worried that doing it this way will lead to bugs when the pointerInput scope is cancelled and restarted on a change in size, which could lead to some pointer events being lost.
  2. Is it generally acceptable to render a larger UI component, like a piano keyboard, entirely using the canvas? I will have animations for key presses, and using a canvas means the entire keyboard will be redrawn on every frame of the animations. Is it common to manually implement some kind of buffer or caching to avoid redrawing components, or is drawing a bunch of simple shapes just so fast that it generally doesn't matter?

Solution

  • First of all if you don't assign any size Modifier to Canvas the size you get from DrawScope will have 0 width and height. In Jetpack Compose as long as you don't need this size param you can draw anywhere without setting it but if you check the log you will see that it will have zero width and height.

    1- You don't need onSizeChanged because PointerInputScope also returns size which is same size unless you set any padding or other layout modifiers after that.

    And if you wish to reset pointerInputScope you can get screen width and height using a BoxWithConstraint as

    @Preview
    @Composable
    private fun Test() {
        BoxWithConstraints(
            modifier = Modifier.fillMaxSize()
        ) {
    
            val width: Dp = maxWidth
            val height: Dp = maxHeight
    
            Canvas(
                modifier = Modifier.background(Color.Black)
                    .pointerInput(width, height) {
                        
                        // The size you get from PointerInputScope is same as you get from 
                        // DrawScope unless you set padding or another layout modifier
                        
                        val size: IntSize = this.size
                        
                        awaitPointerEventScope {
                            /* track pointers and calculate which
                            keys are pressed using the values of
                            width and height */
                        }
                    }
            ) {
                
                val size = this.size
                println("size: $size")
                
                /* draw the piano keys, using the values
                of size.width and size.height to size and
                position them */
            }
            
        }
    }
    

    2- Canvas is a Spacer with Modifier.drawBehind, anything you do in this scope calls only draw phase of composition which is advised by google, especially for animations.

    https://developer.android.com/jetpack/compose/performance/bestpractices#defer-reads

    Check out background animation snippet.