How to make PKCanvasView zoomable in SwiftUI?

There a PKCanvasView setup like this:

struct CanvasView {
  @Binding var canvasView: PKCanvasView
  let onSaved: () -> Void
  @State var toolPicker = PKToolPicker()
}


extension CanvasView: UIViewRepresentable {
  func makeUIView(context: Context) -> PKCanvasView {
      canvasView.tool = PKInkingTool(.pencil, color: .gray, width: 20)
//    #if targetEnvironment(simulator)
      canvasView.drawingPolicy = .anyInput
//    #endif
    canvasView.delegate = context.coordinator
    showToolPicker()
    return canvasView
  }

  func updateUIView(_ uiView: PKCanvasView, context: Context) {}

  func makeCoordinator() -> Coordinator {
    Coordinator(canvasView: $canvasView, onSaved: onSaved)
  }
}

And then I use it in SwiftUI Like this:

        VStack {
            Spacer()
            CanvasView(canvasView: $canvasView, onSaved: saveDrawing)
                .aspectRatio(1.0, contentMode: .fit)
            Spacer()
        }

But the second I put it in the ScrollView to add the pan and zoom ability like this

ScrollView {    //prevents drawing with finger
        VStack {
            Spacer()
            CanvasView(canvasView: $canvasView, onSaved: saveDrawing)
                .aspectRatio(1.0, contentMode: .fit)
            Spacer()
        }
}

It stops allowing me to draw with finger, presuming that it's trying to scroll instead. I'm looking for a behaviour like the Apple's freeform app. Finger to draw and pinch to zoom, 2 fingers to pan.

Updated

After some clarification, I've included the ability to shrink the entire drawing area smaller than the screen size.

Pencil Kit's canvas is robust with built-in zooming and panning. So there's no need to add scroll views or pan gestures, just set the canvas content's size, zoom scales and insets.

For scrolling and panning, set a large canvas content size for the drawing area:

canvasView.contentSize = CGSize(width: 1500, height: 1000)

For zooming, set the desired zoom scales:

canvasView.minimumZoomScale = 0.2
canvasView.maximumZoomScale = 4.0

For allowing the full drawing area to shrink smaller than the scrollview's frame size, add content insets:

canvasView.contentInset = UIEdgeInsets(top: 500, left: 500, bottom: 500, right: 500)

The following is a basic SwiftUI pencil app with toolpicker. This has the pinch-to-zoom, two-finger panning and finger-to-draw:

import SwiftUI
import PencilKit

struct ContentView: View {
    var body: some View {
        CanvasView()
    }
}

struct CanvasView {
    @State var canvasView: PKCanvasView = PKCanvasView()
    @State var toolPicker = PKToolPicker()
}

extension CanvasView: UIViewRepresentable {
    func makeUIView(context: Context) -> PKCanvasView {
            // canvas
        canvasView.contentSize = CGSize(width: 1500, height: 1000)
        canvasView.drawingPolicy = .anyInput
        canvasView.minimumZoomScale = 0.2
        canvasView.maximumZoomScale = 4.0
        canvasView.backgroundColor = UIColor(red: 0.9, green: 0.9, blue: 0.9, alpha: 1.0)
        canvasView.contentInset = UIEdgeInsets(top: 500, left: 500, bottom: 500, right: 500)
        canvasView.becomeFirstResponder()
        
            //toolpicker
        toolPicker.setVisible(true, forFirstResponder: canvasView)
        toolPicker.addObserver(canvasView)
        
        return canvasView
    }
    
    func updateUIView(_ uiView: PKCanvasView, context: Context) {}
}

Note: The gif below was created using the code above on an iPad with the toolpicker moved to the side. The background color(white) was manually painted on the drawing area with the toolpicker's marker to better demonstrate the zooming and panning. Setting the scollview's background color independently of the drawing area is a needed enhancement.

The resulting app looks like this: