Place bibliography before Appendix when using natbib

I have the following document.

---
title: "What comes after the bibliography?"
bibliography: refs.bib
header-includes:
  - \usepackage{natbib}
  - \setcitestyle{numbers}
  - \setcitestyle{square}
output: 
  pdf_document:
    citation_package: natbib
    latex_engine: lualatex
    keep_tex: yes
---

# A

This document may use knitr[@pKnitr].

# References

<div id="refs"></div>

# Appendix

`1+1` evaluates in R to `r 1+1` (for more information, see [@pRmarkdown])

I used citation() to create the bibtex references in refs.bib.

@InCollection{pKnitr,
  booktitle = {{Implementing Reproducible Computational Research}},
  editor = {Victoria Stodden and Friedrich Leisch and Roger D. Peng},
  title = {knitr: A Comprehensive Tool for Reproducible Research in {R}},
  author = {Yihui Xie},
  publisher = {Chapman and Hall/CRC},
  year = {2014},
  note = {ISBN 978-1466561595},
  url = {http://www.crcpress.com/product/isbn/9781466561595}
}

@Book{pRmarkdown,
  title = {{R Markdown Cookbook}},
  author = {Yihui Xie and Christophe Dervieux and Emily Riederer},
  publisher = {Chapman and Hall/CRC},
  address = {Boca Raton, Florida},
  year = {2020},
  note = {ISBN 9780367563837},
  url = {https://bookdown.org/yihui/rmarkdown-cookbook}
}

I want to place my bibliography before the Appendix. So far I've tried the following, unfortunately with no luck.

• Add <div id="refs"></div> (see example above)
• Using \AtEndDocument{} - its contents won't transpile to LaTeX
• \printbibliography does not work with natbib
• \bibliography{refs} kinda works, but I have to manually delete the bibliography macro created by md / Rmd

For the last point, if I have the following in my Rmd file...

<!-- ... -->
# References

\bibliography{refs}

# Appendix

`1+1` evaluates in R to `r 1+1` (for more information, see [@pRmarkdown])

... it produces the following LaTeX file.

% ...
\section{A}\label{a}}

This document may use knitr\citep{pKnitr}.

\hypertarget{references}{%
\section{References}\label{references}}

\bibliography{refs.bib}

\hypertarget{appendix}{%
\section{Appendix}\label{appendix}}

\texttt{1+1} evaluates in R to 2 (for more information, see
\citep{pRmarkdown})

  \bibliography{refs.bib}

\end{document}

So \bibliography{refs.bib} is now added twice, causing a Citations multiply defined error.

How can I add the bibliography before my Appendix using natbib?

If you use remove all the interference from markdown, you are more flexible with the placement of the bibliography:

---
title: "What comes after the bibliography?"
header-includes:
  - \usepackage{natbib}
  - \bibliographystyle{plainnat}
  - \setcitestyle{numbers}
  - \setcitestyle{square}
output: 
  pdf_document:
    latex_engine: lualatex
    keep_tex: yes
---

# A

This document may use knitr \cite{pKnitr}.

\bibliography{refs}

# Appendix

`1+1` evaluates in R to `r 1+1` (for more information, see \cite{pRmarkdown})