Cross referencing pander-table in rmarkdown

Despite the huge amount of questions on this subject and despite having read the manual, including the last paragraph, I can't get my head around it. I found a solution for pdf-output, but would like it to work for html-output too (ideally an approach that works for both pdf- and html-output - like my attempt below using opts_knit$get("rmarkdown.pandoc.to") that sadly fails for html).

Using bookdown, it is easy to reference to kabel(Extra)- or flextable- tables, but my workaround for pander-tables only works for pdf-output. Can anyone guide me on how to get this working with html as well?

The MWE example below works with pdf, but the reference to the last table breaks when outputted to html.

MWE:

---
title: "Untitled"
output:
    bookdown::pdf_document2:
        keep_tex: yes
    bookdown::html_document2: default
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)

library(knitr)
library(tidyverse)
library(pander)
library(flextable)
```


```{r}

df <- expand.grid(A = LETTERS[1:2],
              B = letters[3:5], 
                 stringsAsFactors = FALSE)%>%
              mutate( x = round(rnorm(nrow(.)),1))%>%
              arrange(A,B)


```


```{r mypanderworkaround}

addlabref <- function(fcap, flab){

   if (opts_knit$get("rmarkdown.pandoc.to") %in% 
    c("beamer", "latex")){

    lab <- sprintf("\\label{tab:%s}",flab)

    sprintf("%s %s", lab, fcap)

  }else{

    lab <- sprintf("\\#tab:%s",flab)

    sprintf("<caption>%s %s</caption>", lab, fcap)

  }

}

```


```{r c1}

kable(df, caption = "table with kable")

```

```{r c2}

flextable(df)%>%set_caption(caption = "table with flextable")

```


```{r c3}

pander(df, caption = addlabref("table with pander", "c3") )
```


kable figure is \@ref(tab:c1).

flextable figure is \@ref(tab:c2)

pander figure is \@ref(tab:c3)

This should help:

Our table:

<br>
<div style="text-align: center">
   <caption><span id="tab:c3">Table 3: </span>My table is here</caption>
</div>

<div style="width: 50%; margin: 0 auto;">
```{r c3}
pander(head(mtcars[1:5]))
```
</div>

Our ref:

pander figure is <a href="#tab:c3">3</a>

...looks like

With your table is a same story:

<br>
<div style="text-align: center">
   <caption><span id="tab:c3">Table 3: </span>My table is here</caption>
</div>

<div style="width: 10%; margin: 0 auto;">
```{r c3}
pander(df)
```
</div>

Works everywhere:

```{r c3}
pander(df, caption = "(\\#tab:table-label) Table caption" )
```

pander figure is \@ref(tab:table-label)