I have been able to register my own mach port to capture mach exceptions in my applications and it works beautifully when I target 32 bit. However when I target 64 bit, my exception handler catch_exception_raise() gets called but the array of exception codes that is passed to the handler are 32 bits wide. This is expected in a 32 bit build but not in 64 bit.
In the case where I catch EXC_BAD_ACCESS the first code is the error number and the second code should be the address of the fault. Since the second code is 32 bits wide the high 32 bits of the 64 bit fault address is truncated.
I found a flag in <mach/exception_types.h> I can pass in task_set_exception_ports() called MACH_EXCEPTION_CODES which from looking at the Darwin sources appears to control the size of the codes passed to the handler. It looks like it is meant to be ored with the behavior passed in to task_set_exception_ports().
However when I do this and trigger an exception, my mach port gets notified, I call exc_server() but my handler never gets called, and when the reply message is sent back to the kernel I get the default exception behavior.
I am targeting the 10.6 SDK.
I really wish apple would document this stuff better. Any one have any ideas?
Well, I figured it out.
To handle mach exceptions, you have to register a mach port for the exceptions you are interested in. You then wait for a message to arrive on the port in another thread. When a message arrives, you call exc_server() whose implementation is provided by System.library. exec_server() takes the message that arrived and calls one of three handlers that you must provide. catch_exception_raise(), catch_exception_raise_state(), or catch_exception_raise_state_identity() depending on the arguments you passed to task_set_exception_ports(). This is how it is done for 32 bit apps.
For 64 bit apps, the 32 bit method still works but the data passed to you in your handler may be truncated to 32 bits. To get 64 bit data passed to your handlers requires a little extra work that is not very straight forward and as far as I can tell not very well documented. I stumbled onto the solution by looking at the sources for GDB.
Instead of calling exc_server() when a message arrives at the port, you have to call mach_exc_server() instead. The handlers also have to have different names as well catch_mach_exception_raise(), catch_mach_exception_raise_state(), and catch_mach_exception_raise_state_identity(). The parameters for the handlers are the same as their 32 bit counterparts.
The problem is that mach_exc_server() is not provided for you the way exc_server() is. To get the implementation for mach_exc_server() requires the use of the MIG (Mach Interface Generator) utility. MIG takes an interface definition file and generates a set of source files that include a server function that dispatches mach messages to handlers you provide. The 10.5 and 10.6 SDKs include a MIG definition file <mach_exc.defs> for the exception messages and will generate the mach_exc_server() function. You then include the generated source files in your project and then you are good to go.
The nice thing is that if you are targeting 10.6+ (and maybe 10.5) you can use the same exception handling for both 32 and 64 bit. Just OR the exception behavior with MACH_EXCEPTION_CODES when you set your exception ports. The exception codes will come through as 64 bit values but you can truncate them to 32 bits in your 32 bit build.
I took the mach_exc.defs file and copied it to my source directory, opened a terminal and used the command mig -v mach_exc.defs. This generated mach_exc.h, mach_excServer.c, and mach_excUser.c. I then included those files in my project, added the correct declaration for the server function in my source file and implemented my handlers. I then built my app and was good to go.
Well, this not the best description, but hopefully it helps someone else out.