Interacting with view makes toolbar & title disappear

I'm developing a Watch app that needs to receive images from iPhone. Users of the app need their hands free, which is why they will have to inspect those images on Watch. Thus, I added a rudimentary image viewer, allowing them to zoom in on images and pan around.

As soon as the user pans or zooms, toolbar and title will disappear. I observed this behavior on simulator and my colleague's Watch (A2376), but not on mine (A2356). Tested both on watchOS 9.6.2 and 10.0.1.

Why is this happening?

Here's some exemplar code:

// View is presented on a sheet

TabView(selection: $viewing) {
    ForEach(images) { reference in
        reference.img
            .resizable()
            .aspectRatio(contentMode: .fit)
            .scaleEffect(scale)
            .offset(offset + draggingOffset)
            .frame(width: screenSize.width, height: screenSize.height-1)
            .clipped()
            .id(reference.id)
            .gesture(panGesture, including: .gesture)
            .gesture(tapZoomGesture)
            .navigationTitle(viewerTitle)
            .focusable()
            .digitalCrownRotation(detent: $scale,
                                  from: 0.5,
                                  through: scaleMax,
                                  by: 0.1,
                                  sensitivity: .low,
                                  isContinuous: false,
                                  isHapticFeedbackEnabled: false,
                                  onIdle: { offset = stickyEdgeOffset })
    }
}

The solution was lifting out navigationTitle and toolbar. I really got lucky guessing.. can't imagine it's supposed to work like this.

someView
  .sheet(isPresented: $isShowingViewer, content: {
    imageViewer
      .navigationTitle("Image viewer")
      .toolbar(content: {
          ToolbarItem(placement: .confirmationAction, content: {
              imageViewerToolbar
        })
    })
})