How do I use a pager for long git add --patch hunks?

When I interactively add diff hunks with git add --patch, I sometimes get hunks which are longer than the screen, but no ability to use less to page through the hunks.

This is strange to me as I have already set:

[core]
      pager = less -FRX --tabs=4

[pager]

  diff = diff-highlight | less -FRX --tabs=4

interactive.diffFilter= piped through less doesn't help with paging either.

What do I need to do to get git add--patch to use less such that I can use the keyboard to navigate any output longer than one screen?

Based on AtnNn's answer, I came up with the following alias:

ap = !"VISUAL=\"VISUAL='$VISUAL' bash -c 'set -m; less -K -+F \\\"\\$1\\\"' --\" git add -p \"$@\" #"

• Press e to page the hunk in less
  • Press q to stage what was displayed
  • Press v to edit what is displayed
  • Press ^C to quit and repeat the interactive menu

I'm working on a PR to fix this in git itself.