How can I make a button not fire its action on click-through?

The Apple Human Interface Guidelines state that:

An item that provides click-through is one that a user can activate on an inactive window with one click, instead of clicking first to make the window active and then clicking the item. Click-through provides greater efficiency in performing such tasks as closing or resizing inactive windows, and copying or moving files. In many cases, however, click-through could confuse a user who clicks an item unintentionally.

and

Don’t provide click-through for an item or action that:

• Is potentially harmful and does not allow the user to cancel it (for example, the Delete button in Mail)
• Is difficult or impossible to cancel (such as the Send button in Mail)
• Dismisses a dialog without telling the user what action was taken (for example, the Save button in a Save dialog that overwrites an existing file and automatically dismisses the dialog)
• Removes the user from the current context (for example, selecting a new item in a Finder column can change the target of the Finder window)

What I want to do is that if the user clicks a specific button it will not send its message unless the window is active (for example, the delete message button in Mail). How can I achieve this? If I need to subclass NSButton that's fine.

Look at the NSView Documentation:

http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/ApplicationKit/Classes/NSView_Class/Reference/NSView.html#//apple_ref/occ/instm/NSView/acceptsFirstMouse:

You need to override acceptsFirstMouse to return TRUE to enable click through.

The default behaviour is not click-through:

The default implementation ignores theEvent and returns NO.

It's possible you have already overridden this method in your code, or in code you have based your code on. Try removing the implementation of acceptsFirstMouse in your code.