I have my main dune project, which includes an implementation for a small parser library. Ideally, I would like to put the parser library into a different package and just depend on that in my main dune project somehow.
The way I would do this in Haskell with stack would be to put my parser library in a separate git repo and then include that git repo as a dependency in my main project.
I haven't found a way to depend on git projects in dune/opam though. Is the only way to include external packages really to publish them to the opam repository?
You can include dune subprojects into your project using vendoring. The only limitation is that the dependency has to be a dune project itself (i.e., have the dune file).
If you have many vendored dependencies, you might find the opam-monorepo plugin useful. This plugin will help you manage them but, of course, it will require that the dependencies are dune projects and have the opam file (the latter is not that constraining, as you can always generate an opam file from the dune file). It is important to notice, that with opam monorepo your project will, in the end, be independent of opam or opam dependencies. Essentially, opam monorepo will just help you vendor all the dependencies into your project so that later you can build everything from the same source tree.