I have contributions to make to FOSS repository that add significant scope. At what point is it preferred to create a diverging standalone repository, as opposed to a branch + pull request, when adding new functionality to a FOSS repo? What are the best practices here?
The most obvious way to add new features to a repo would be to contribute via a new branch + PR, giving the maintainers the opportunity to merge your expansion into the project's master branch. The downside being that contributions add up, and the scope/complexity will increase in directions perhaps not originally desired. Maintainers obviously have to be careful what contributions are pulled into the repo.
In an effort to, as the contributor, be respectful of the software's maintainers and original vision, when should one create a standalone fork of the project with intention to develop it in parallel, as opposed to a branch + PR? When the changes are breaking? When the contribution is in a new direction? When the features would be hard to maintain? Should a new repo only be created if a PR is denied? etc.
For example: many people fork Linux distros, make tiny changes, and have their own personalized distro, instead of adding their changes as PR's to the source distro.
Edit: reworked usage of "pull request" to reflect the fact that it is essentially a fork+merge
I think you're fundamentally misunderstanding how github (and FOSS development) works.
A pull request always goes from one branch to another, i.e. from the branch that has your changes to some branch that has the development version of the software you're contributing to (some repos use the main branch for that, others have one or more development branches). Read the docs and/or talk to the maintainers of the software to find out which branch is the right one and what requirements your PR should fulfill.
In Github, you can also do cross-repository pull requests, with a branch in your fork as origin and a branch in the original (aka "upstream") repository as target.
Since almost all FOSS projects restrict directly committing to the repository to trusted contributors (and to avoid cluttering the main repo in all other cases), you should always create your own fork, do your changes there, and then create the PR to bring the contributions upstream.
I tend to keep most those forks as private repositories, as others don't need to see my half-finished stuff. But feel free to make the fork public, if you think it's worth it for someone else to see it. Since we're talking about FOSS, that shouldn't be an issue with licensing - if you aren't allowed to make a public fork, it's not FOSS. Be mindful of trademarks, though. Some FOSS has trademarked names associated with it, that you aren't free to copy.
If you're fixing bugs or adding features, it's always worth it to talk to the upstream maintainers about whether they want to incorporate it - that's one use of a pull request. If they don't want it, they can reject the PR.
If you're planning massive changes to existing functionality, it's usually a good idea to talk to the upstream maintainers about your plans before starting. Maybe they're enthusiastic and want to join in, maybe they already tried something similar and can tell you why it didn't work out, and maybe they're open to part, but not all of it.
If your version starts to diverge significantly because of changes that won't be merged into upstream (e.g. if you're doing a complete redesign that the upstream maintainers don't like), it's good manners to choose a different name for your software, in order to prevent confusion about which is which.
Never feel bad about making a fork. That's one of the most FOSS things you can do.