apachelinkerclangllvm

Compiling Apache Server using LLVM Plus a Custom Pass and a Custom Library


We would like to use LLVM to compile apache 2.2.19. In the compilation process, we want to add a custom pass to clang by specifying the CFLAGS environment variable:

CFLAGS="-Xclang -load -Xclang mypass.so"

Then, at the linking step, we also want the executable to be linked with one custom dynamic library. So we exported these two environment variables:

LDFLAGS="-L. -lmylib"
LD_LIBRARY_PATH='./libfolder'

We've used these environment variables when compiling MySQL, and it worked as expected. However, for compiling apache, we encountered the following error:

/usr/bin/ld: cannot find -load

It seems that the the linker also reads compiler flags and then interpreted -load as a library... Could you please tell me how I can get rid of this issue?

Below are more details:

Here is the command I used to configure apache:

CC=clang CXX=clang++
CFLAGS="-Xclang -load -Xclang /home/my/software/llvm-build/Debug+Asserts/lib/mypass.so" 
CXXFLAGS="-Xclang -load -Xclang /home/my/software/llvm-build/Debug+Asserts/lib/mypass.so"
LDFLAGS="-ldl -L/home/my/Projects/lib/ -lmylib"
LD_LIBRARY_PATH="/home/my/Projects/lib/"
./configure

The error after I ran make is here:

/bin/bash /home/my/Downloads/httpd-2.2.19/srclib/apr/libtool --silent --mode=link
clang -Xclang -load -Xclang /home/my/software/llvm-build/Debug+Asserts/lib/mypass.so 
-DHAVE_CONFIG_H -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
-I./include -I/home/my/Downloads/httpd-2.2.19/srclib/apr/include/arch/unix
-I./include/arch/unix -I/home/my/Downloads/httpd-2.2.19/srclib/apr/include/arch/unix
-I/home/my/Downloads/httpd-2.2.19/srclib/apr/include  -version-info 4:5:4   --verbose -ldl
-L/home/my/Projects/lib/ -lmylib-o libapr-1.la
-rpath /usr/local/apache2/lib passwd/apr_getpass.lo strings/apr_cpystrn.lo
strings/apr_fnmatch.lo strings/apr_snprintf.lo strings/apr_strings.lo
strings/apr_strnatcmp.lo strings/apr_strtok.lo tables/apr_hash.lo
tables/apr_tables.lo atomic/unix/builtins.lo atomic/unix/ia32.lo
atomic/unix/mutex.lo atomic/unix/ppc.lo atomic/unix/s390.lo
atomic/unix/solaris.lo dso/unix/dso.lo file_io/unix/buffer.lo
file_io/unix/copy.lo file_io/unix/dir.lo file_io/unix/fileacc.lo file_io/unix/filedup.lo
file_io/unix/filepath.lo file_io/unix/filepath_util.lo file_io/unix/filestat.lo
file_io/unix/flock.lo file_io/unix/fullrw.lo file_io/unix/mktemp.lo file_io/unix/open.lo
file_io/unix/pipe.lo file_io/unix/readwrite.lo file_io/unix/seek.lo
file_io/unix/tempdir.lo locks/unix/global_mutex.lo locks/unix/proc_mutex.lo
locks/unix/thread_cond.lo locks/unix/thread_mutex.lo locks/unix/thread_rwlock.lo
memory/unix/apr_pools.lo misc/unix/charset.lo misc/unix/env.lo misc/unix/errorcodes.lo
misc/unix/getopt.lo misc/unix/otherchild.lo misc/unix/rand.lo misc/unix/start.lo
misc/unix/version.lo mmap/unix/common.lo mmap/unix/mmap.lo network_io/unix/inet_ntop.lo
network_io/unix/inet_pton.lo network_io/unix/multicast.lo network_io/unix/sendrecv.lo
network_io/unix/sockaddr.lo network_io/unix/socket_util.lo network_io/unix/sockets.lo
network_io/unix/sockopt.lo poll/unix/epoll.lo poll/unix/kqueue.lo poll/unix/poll.lo
poll/unix/pollcb.lo poll/unix/pollset.lo poll/unix/port.lo poll/unix/select.lo
random/unix/apr_random.lo random/unix/sha2.lo random/unix/sha2_glue.lo shmem/unix/shm.lo
support/unix/waitio.lo threadproc/unix/proc.lo threadproc/unix/procsup.lo
threadproc/unix/signals.lo threadproc/unix/thread.lo threadproc/unix/threadpriv.lo
time/unix/time.lo time/unix/timestr.lo user/unix/groupinfo.lo user/unix/userinfo.lo
-luuid -lrt -lcrypt  -lpthread

/usr/bin/ld: cannot find -load
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [libapr-1.la] Error 1
make[3]: Leaving directory `/home/my/Downloads/httpd-2.2.19/srclib/apr'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/my/Downloads/httpd-2.2.19/srclib/apr'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/my/Downloads/httpd-2.2.19/srclib'
make: *** [all-recursive] Error 1

Solution

  • I figured out a solution, although I am not 100% sure about the root cause.

    It seems that apache's configure file will put compiler flags and linker flags together when linking. So what we can do is to find all link related statements in configure. For example, we use the following command in the root directory:

    grep -r '\--mode=link' .
    

    Then we go to each configure file and remove $(COMPILE)

    You might also encounter tag issue, which can be solved in this way: https://forums.gentoo.org/viewtopic-t-915572-start-0.html

    Also, since we are linking a dynamic library, we need to export the correct LD_LIBRARY_PATH before run make.