I am trying to find out which Hypervisor will allow me to grant access to specialized PCI cards (such as a telephony card) to a virtual machine. So far I have tried out VMWare ESXi server and it doesn't seem to allow me to do this. I have heard that Microsoft Virtual Server does allow this, but I haven't been able to find any supporting documentation.
I'd look into Xen, it appears that you can load a backend xen driver on the host OS which will then allow you to communicate directly with the hardware from the guest.
See this link for more information. I'm not a Xen user, but from my virtualization experience I would guess that the paravirtualization aspects of a Xen host/guess is going to be your best bet for raw device access.