I have a command-line java program that I will need to call from a kernel mode driver (a MiniFilter Driver on Windows).
Is it possible ? Is there somethings I will have to take care to avoid problems ? How can I do this ?
The java program runs in user-mode. You can't just make a function call from kernel-mode to user-mode. But you can communicate between the two.
Usually the user-mode program invokes the functionality of the driver by calling a user-mode IO API on a handle opened on a device object associated with the driver. There are standard IO operations like read and write (i.e. calling Win32 APIs ReadFile() or WriteFile() in C/C++), but you can also implement your own custom/proprietary operations using DeviceIoControl(). You pass an operation code and a pair of buffers: one with arguments/input to the driver, and one to receive its output. On top of this you could build a mechanism for the user-mode program to request "calls" or messages from the driver, and the driver wouldn't complete a request until it wanted to send such a call/message. It would pass its arguments in the output buffer of a pending DeviceIoControl request. Then the program could use another DeviceIoControl() call, possibly with a different operation code, to return the results to the driver, in the input buffer.
You have the added burden invoking DeviceIoControl() from Java. For that you may need to use JNI and a bit of C/C++ code in a DLL.
Beware gotchas of having a driver depend on a user-mode program. You probably don't want to implement a blocking call in the driver that waits on the response from user-mode because the user-mode program may crash or be killed and never complete the call the driver is waiting on, and then you have a hung thread, probably in some process other than your java program's. Even if you implement asynchronous completion of calls sent to user-mode, you probably want your calls to have a timeout mechanism, so that if the user-mode program doesn't complete the call, you can clean up any related context, such as a call identifier in a collection of pending calls. Also look out for problems similar to those in multi-threading or multi-tasking, such as deadlock. Also note data passed from kernel-mode to user-mode usually should be copied into the user-mode output buffer, and not contain any pointers to virtual addresses in the kernel address space or the address space of other processes.