We are in the middle of choosing our headless browser driver solution that will be some implementation of Selenium WebDriver. There is the GhostDriver, which leverages the PhantomJS in the backend on the one side and HtmlUnitDriver which based on HtmlUnit on the other.
PhantomJS uses WebKit, the rendering engine of Safari, to render the pages while HtmlUnitDriver uses the Rhino engine which no other browsers use (it's just "simulating" browser behaviour. The last fact considered as a con, because the rendering behavior can differ significantly from the popular browsers.
In our opinion, PhantomJS is a much stronger candidate. But, we don't know everything :) Is there other considerations and trade-offs we should take into account with our decision? other scenarios where HtmlUnitDriver can be a better solution?
From my experience with a number of headless browsers, I'd say:
HtmlUnitDriver: the fastest of all implementations I've come across, and perfect for simple, static pages, especially those without JavaScript. Any remotely complex page seems to produce problems - that's my practical experience even if I can't justify in detail. Perfect for testing Selenium features on demo pages, scraping status pages etc.
PhantomJSDriver (PhantomJS + GhostDriver): not as much faster as you might hope vs the desktop browsers, however, much easier to set up than Firefox + xvfb. By default screenshots can look a bit odd, but that usually turns out to be because PhantomJS defaults to a narrow window unless explicitly set (read below for why).
Update: a bit more detail on PhantomJS versions, from my other answer.
Like Safari, PhantomJS uses WebKit for rendering (e.g. Firefox uses Gecko)
Different PhantomJS versions are built against different WebKit versions. PhantomJS 2.x uses WebKit 538.x, which makes it equivalent to Safari 7 or 8. whereas PhantomJS 1.9.8 uses WebKit 534.34, which is equivalent to Safari 5.
This may be an issue for you, since Google determines Safari 5 to be an "old" browser and will therefore potentially render its search pages differently.
So ensuring you use PhantomJS 2.x can reduce the rendering differences that a lot of people report vs. desktop browsers.
Another interesting possibility is SlimerJS. However, I've not got it to work reliably enough yet.
I've never had reliability issues with either HtmlUnitDriver or PhantomJSDriver (the only annoyance one was a HttpClient 4.5 / HtmlUnit 2.17 incompatibility issue).
(In answer to the comment about modifying HTTP requests, I'd personally recommend sticking to the WebDriver API and use a proxy like BrowserMob to mutate requests and responses rather than taking advantage of browser-specific features.)
All in all, I'd advise against creating a tool or process that forces users to choose one browser over another. If possible, allow them to configure or override. For the majority of cases I'd plump for PhantomJS, as it won't let you down. However, the performance of HtmlUnit should be considered for the simplest pages.
See also (perhaps): http://www.guru99.com/selenium-with-htmlunit-driver-phantomjs.html and https://www.quora.com/Software-Testing/How-does-PhantomJS-compare-to-Selenium