I want to install some software like a MySQL CLI tool and the docs say .ddev/config.yaml can be edited to add extra debian packages... but what about non-debian packages? How do you install something that would normally be installed via pip install -U mycli?
The docs cover much of this. The approach here is also written up more extensively in Customizing DDEV images with a custom Dockerfile]
The mysql CLI tool (mysql) is already installed in both the ddev web and db containers, but you definitely like mycli.
As you know, most popular Linux tools have Debian packages, and it's easy to install those using webimage_extra_packages: [package1, package2] per https://ddev.readthedocs.io/en/stable/users/extend/customizing-images/#adding-extra-debian-packages-with-webimage_extra_packages-and-dbimage_extra_packages
In your case, mycli is distributed as a Debian package per its github page, so you can just use webimage_extra_packages: [mycli], and that's the best way to install it.
But to automatically install other tools when the container is built (which use npm or pip3 or pecl/pear or whatever) you'd use a custom Dockerfile. We'll use your example of mycli (as if it didn't have a Debian package).
In .ddev/web-build/Dockerfile you'd want something like this, based on .ddev/web-build/Dockerfile.example, which first installs pip3 and then uses that to install mycli:
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends --no-install-suggests python3-pip python3-setuptools
RUN pip3 install mycli
Of course for npm packages we could take the example from the Dockerfile.example, RUN npm install --global gulp-cli. And it's even possible to install gcc or g++ (just as we installed pip3) and build a package from source.
If you wanted to install a PECL/Pear package, the approach in https://stackoverflow.com/a/60554990/215713 will be work. It's the same basic idea.
The great thing about the custom Dockerfile (or webimage_extra_packages, when it's needed, is that the Docker image containing the extra instructions only needs to be built once per configuration, not on every startup. Note though that a custom Dockerfile overrides webimage_extra_packages, so they're mutually exclusive.