Mixing PIC and non-PIC objects in a shared library

This question is related to this one as well as its answer.

I just discovered some ugliness in a build I'm working on. The situation looks somewhat like the following (written in gmake format); note, this specifically applies to a 32-bit memory model on sparc and x86 hardware:

OBJ_SET1  := some objects
OBJ_SET2  := some objects

# note: OBJ_SET2 doesn't get this flag
${OBJ_SET1} : CCFLAGS += -PIC

${OBJ_SET1} ${OBJ_SET2} : %.o : %.cc
  ${CCC} ${CCFLAGS} -m32 -o ${@} -c ${<}

obj1.o       : ${OBJ_SET1}
obj2.o       : ${OBJ_SET2}
sharedlib.so : obj1.o obj2.o
obj1.o obj2.o sharedlib.so :
  ${LINK} ${LDFLAGS} -m32 -PIC -o ${@} ${^}

Clearly it can work to mix objects compiled with and without PIC in a shared object (this has been in use for years). I don't know enough about PIC to know whether it's a good idea/smart, and my guess is in this case it's not needed but rather it's happening because someone didn't care enough to find out the right way to do it when tacking on new stuff to the build.

My question is:

1. Is this safe
2. Is it a good idea
3. What potential problems can occur as a result
4. If I switch everything to PIC, are there any non-obvious gotchas that I might want to watch out for.

Forgot I even wrote this question.

Some explanations are in order first:

• Non-PIC code may be loaded by the OS into any position in memory in [most?] modern OSs. After everything is loaded, it goes through a phase that fixes up the text segment (where the executable stuff ends up) so it correctly addresses global variables; to pull this off, the text segment must be writable.
• PIC executable data can be loaded once by the OS and shared across multiple users/processes. For the OS to do this, however, the text segment must be read-only -- which means no fix-ups. The code is compiled to use a Global Offset Table (GOT) so it can address globals relative to the GOT, alleviating the need for fix-ups.
• If a shared object is built without PIC, although it is strongly encouraged it doesn't appear that it's strictly necessary; if the OS must fix-up the text segment then it's forced to load it into memory that's marked read-write ... which prevents sharing across processes/users.
• If an executable binary is built /with/ PIC, I don't know what goes wrong under the hood but I've witnessed a few tools become unstable (mysterious crashes & the like).

The answers:

• Mixing PIC/non-PIC, or using PIC in executables can cause hard to predict and track down instabilities. I don't have a technical explanation for why.
  • ... to include segfaults, bus errors, stack corruption, and probably more besides.
• Non-PIC in shared objects is probably not going to cause any serious problems, though it can result in more RAM used if the library is used many times across processes and/or users.

---

update (4/17)

I've since discovered the cause of some of the crashes I had seen previously. To illustrate:

/*header.h*/
#include <map>
typedef std::map<std::string,std::string> StringMap;
StringMap asdf;

/*file1.cc*/
#include "header.h"

/*file2.cc*/
#include "header.h"

int main( int argc, char** argv ) {
  for( int ii = 0; ii < argc; ++ii ) {
    asdf[argv[ii]] = argv[ii];
  }

  return 0;
}

... then:

$ g++ file1.cc -shared -PIC -o libblah1.so
$ g++ file1.cc -shared -PIC -o libblah2.so
$ g++ file1.cc -shared -PIC -o libblah3.so
$ g++ file1.cc -shared -PIC -o libblah4.so
$ g++ file1.cc -shared -PIC -o libblah5.so

$ g++ -zmuldefs file2.cc -Wl,-{L,R}$(pwd) -lblah{1..5} -o fdsa
#     ^^^^^^^^^
#     This is the evil that made it possible
$ args=(this is the song that never ends);
$ eval ./fdsa $(for i in {1..100}; do echo -n ${args[*]}; done)

That particular example may not end up crashing, but it's basically the situation that had existed in that group's code. If it does crash it'll likely be in the destructor, usually a double-free error.

Many years previous they added -zmuldefs to their build to get rid of multiply defined symbol errors. The compiler emits code for running constructors/destructors on global objects. -zmuldefs forces them to live at the same location in memory but it still runs the constructors/destructors once for the exe and each library that included the offending header -- hence the double-free.