How can I scroll in both directions in Jetpack Compose

I've created a fairly classic collapsing image layout in Jetpack compose, where I have an image at the top of the screen which parallax scrolls away and at a certain point I change the toolbar background from transparent to primarySurface. This is all working quite nicely.

I now want to have a pager of images at the top instead of a single one but the vertical scroll is consuming all the touches in the top part of the screen. I've tried adding a NestedScrollConnection but I still only seem to get the preScroll delta on one axis. I apparently can't even add icons within this area to do a manual pager scroll without the click being consumed. As soon as I remove the verticalScroll from my Column I'm able to get the horizontal scroll events for the pager.

I'm using Compose 1.0.2 and Accompanist 0.18 pager and inset libraries.

This is a gist of the existing code that I want to add a pager into. How can I get both the pager and the vertical scroll to work?

val scrollState = rememberScrollState()
Box {
    val imageHeight =
        if (LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE) 180.dp else 300.dp
    Box {
        // I want to insert a horizontal pager here
        HeaderImage(
            scrollPosition = scrollState.value,
            info = item.heroImages.first(),
            imageHeight = imageHeight
        )
    }
    val appBarHeight = with(LocalDensity.current) { 56.dp.toPx() }
    val scrollHeight = with(LocalDensity.current) { imageHeight.toPx() - appBarHeight }

    Column(
        Modifier
            .verticalScroll(scrollState)
            .padding(top = imageHeight)
            .fillMaxSize()
    ) { ... }
    TopAppBar( ... )

Here is a github repository where I've tried to show a simple example of the issue. I either end up with a horizontal pager that can't be scrolled vertically or where the main scroll layout also consumes the horizontal scrolls that the pager requires in order to work.

https://github.com/barry-irvine/scroll_issue

There is no problem with placing a horizontal scroll view inside a vertical one: scrolling will work without problems, and the current scrolling direction of the gesture will be chosen based on the first dragged pixels direction.

Column(
    Modifier
        .verticalScroll(scrollState)
) {
    HorizontalPager(/*...*/)
    OtherScrollableContent(/*...*/)
}