Consider this scenario in which an executable A.bin uses libY.so and libZ.so. A.c, Y.c and Z.c are all written in C. Z.c and Y.c are compiled into respective .so files.
This is the directory structure of the files
$home/bin/A.bin $home/lib/libY.so $home/lib/libZ.so
When I run A.bin as normal user, A.bin runs normally as expected. Note: $LD_LIBRARY_PATH contains $home/lib
I changed some code in A.c adding some functionality which needs admin privileges(like binding to a port less than 1000). I set the setuid bit for A.bin, libY.so and libZ.so to rwsrwsrws, and change the ownership of the files to root. When I try to run A.bin, I get the following error
ld.so.1: A.bin: fatal: libY.so: open failed: No such file or directory Killed
When I just remove the setuid permission from all those files, then the binary runs except for the functionality fails where it needs root privileges.
How to overcome this problem ?
Edit: The OS is Solaris 5.10
As AProgrammer said, while executing setuid programs, $LD_LIBRARY_PATH is ignored. Hence the path has to be hardcoded in the executable itself using this flag while linking
gcc -R $home/lib
The -R flag builds runtime search path list into executable.