Is it possible to force gcc use int instruction for all the system calls, but not sysenter? This question may sound strange but I have to compile some projects like Python and Firefox this way.
Summary
Thanks to jbcreix, I've downloaded glibc 2.9 source code, and modified the lines in sysdeps/unix/sysv/linux/i386/sysdep.h, to disable use of sysenter by #undef I386_USE_SYSENTER
, and it works.
Recompile your C library after replacing sysenter
by int 0x80
in syscall.s
and link again.
This is not compiler generated code which means you are lucky.
The ultimate origin of the actual syscall is here, as the OP says:
http://cvs.savannah.gnu.org/viewvc/libc/sysdeps/unix/sysv/linux/i386/sysdep.h?root=libc&view=markup
And as I suspected there really was a syscall.S it's just that the glibc sources are a labyrinth.
http://cvs.savannah.gnu.org/viewvc/libc/sysdeps/unix/sysv/linux/i386/syscall.S?root=libc&view=markup
So I think he got it right, asveikau.