Linux supports sending an arbitrary Posix-Signal such as SIGINT or SIGTERM to a process using the kill-Command. While SIGINT and SIGTERM are just boring old ways to end a process in a friendly or not-so-friendly kind of way, SIGQUIT is meant to trigger a core dump. This can be used to trigger a running Java VM to print out a thread dump, including the stacktraces of all running threads -- neat! After printing the debugging info, the Java VM will continue doing whatever it was doing before; in fact the thread dump just happens in another spawned thread of maximum priority. (You can try this out yourself by using kill -3 <VM-PID>.)
Note that you can also register your own signal handlers using the (unsupported!) Signal and SignalHandler classes in the sun.misc-package, so you can have all kinds of fun with it.
However, I have yet to find a way to send a signal to a Windows process. Signals are created by certain user inputs: Ctrl-C triggers a SIGINT on both platforms, for instance. But there does not seem to be any utility to manually send a signal to a running, but non-interactive process on Windows. The obvious solution is to use the Cygwin kill executable, but while it can end Windows processes using the appropriate Windows API, I could not send a SIGBREAK (the Windows equivalent to SIGQUIT) with it; in fact I think the only signal it is able to send to Windows processes is SIGTERM.
So, to make a long story short and to repeat the headline: How to I send an arbitrary signal to a process in Windows?
If what you want is to explicitly/programmatically kill another program/process of any kind, within the SysInternals' pstools there is a small tool named "pskill" that behaves just like Unixen "kill" would do.
If you want something else, keep reading (though I may be wrong on some of the specifics below - it's been eons since I last developed a Windows program in C using only the WinAPI and Charles Petzold's excellent books "Programming for Windows" as a guide).
On Windows you don't properly have "signals", what functions WinMain and WinProc receive from the Operating System are simple messages. For instance, when you click on the "X" button of a window, Windows sends that windows' handler the message WM_CLOSE. When the window's deleted but program's still running, it sends WM_DESTROY. When it's about to get out of the main message processing loop, WinMain (not WinProc) receives WM_QUIT. Your program should respond to all these as expected - you can actually develop an "unclosable" application by not doing what it should upon receiving a WM_CLOSE.
When user selects the task from Windows Task Manager and clicks "End Task", the OS will send WM_CLOSE (and another one I don't remember). If you use "End Process", though, the process is killed directly, no messages sent ever (source: The Old New Thing)
I remember there was a way to get the HWND of another process' window, once you get that another process could send that window a message thru functions PostMessage and DispatchMessage.