I'm referring to OpenSSL as an example but I guess my question as well applies to similarly configured/built libraries.
In order to avoid unnecessary rebuilds (and to track upstream changes and build cache misses) I want to build OpenSSL in a 'target agnostic' way. I.e. OpenSSL will be bundled with a parent project and at build time the final installation target is not known yet.
Configure accepts an argument --prefix which points to an installation destination, as well as --openssldir which together define the directory structure where you can find e.g. openssl.cnf
# --prefix prefix for the OpenSSL installation, which includes the
# directories bin, lib, include, share/man, share/doc/openssl
# This becomes the value of INSTALLTOP in Makefile
# (Default: /usr/local)
# --openssldir OpenSSL data area, such as openssl.cnf, certificates and keys.
# If it's a relative directory, it will be added on the directory
# given with --prefix.
# This becomes the value of OPENSSLDIR in Makefile and in C.
# (Default: PREFIX/ssl)
Unfortunately (I guess for security reasons) --prefix has to be absolute (also see this question).
If I now don't provide --prefix the build directory will be compiled into the final binaries, resulting in error messages like
$ openssl req
Can't open /ci/.cache/bazel/_bazel_ci/c4987ee30ca479718f1c230dbaf68e70/sandbox/processwrapper-sandbox/2/execroot/my_project/bazel-out/k8-fastbuild/bin/external/openssl/openssl.build_tmpdir/openssl/skel/etc/ssl/openssl.cnf for reading, No such file or directory
Is there a way to either configure libraries like OpenSSL to search for files in a relative manner or to 're-configure' the readily built artifacts to live inside a directory only known on installation? How do others do this?
In order to avoid unnecessary rebuilds (and to track upstream changes and build cache misses) I want to build OpenSSL in a 'target agnostic' way.
This is not possible in general. There may be approaches that satisfy you if you can sufficiently limit the range of targets you want to support.
I.e. OpenSSL will be bundled with a parent project and at build time the final installation target is not known yet.
Making it work specifically for a particular application or applications with which it is bundled, without regard for others not included in the bundle, is another limitation that opens possibilities.
If I now don't provide
--prefixthe build directory will be compiled into the final binaries
Are you sure? If you, as builder, do not specify a --prefix explicitly then the default is /usr/local, not the build directory or anything related to it. The help text you quoted even says so. Perhaps you are getting an explicit --prefix from bazel. That would be specific to your particular builds, and presumably under your control. Or perhaps the result of your experiment just doesn't mean what you think it means. The strings utility can help you determine whether the build path is in fact recorded in your built artifacts.
Is there a way to either configure libraries like OpenSSL to search for files in a relative manner or to 're-configure' the readily built artifacts to live inside a directory only known on installation?
Whether and how libraries support dynamic discovery of their runtime configuration or data resources depends on the library. There is no convention for this. Compiled binaries typically do not have a way to determine their installation location at runtime, however, so that particular source of information is probably off the table for OpenSSL.
As for reconfiguration, to a pretty good approximation, you need to rebuild binary artifacts to change their contents, such as to alter paths stored within. You might be able to avoid re-running configure (for example, you can override the prefix on the make command line), but you cannot, generally, avoid a rebuild.
How do others do this?
Generally speaking, others approach similar problems either by
avoiding it.
That can be done by requiring installation in a location consistent with build-time configuration. In your case, that might also require ensuring that a suitable --prefix is in fact expressed at configure time.
making it someone else's responsibility.
That basically boils down to not bundling third-party libraries with your software, or at least providing a way for the libraries provided by the installation host to be used instead.
accepting that the software will need to be rebuilt to accommodate different paths to needed resources.
That amounts to not providing pre-built versions of the affected binaries, but instead building them in situ. This is the traditional way, and it is still relatively common.
Providing separate binary distributions for multiple host types, each configured suitably for hosts of that type.
providing for some kind of runtime configuration, possibly
programmatic, where the client specifies appropriate paths via constructor argument or function call.
OpenSSL in particular is pretty flexible about allowing the application to load overall configuration from a path it specifies, to set specific configration details programmatically, and similar. Appropriate use of such features can make any compile-time default paths moot.
via command-line argument (which may break down to the previous in practice), or
via environment variable, or
relative to dynamically-determined installation location,
etc.
Which options are available to which projects depends on project details.
Overall, constructing a software bundle is usually more complicated than just building all the components and throwing them together. You typically need to configure them specifically for the target bundle, subject to limitations that imposes, and you may need to write or modify them specifically for such a context.