I'm just wondering is it stupid to use both Xcache 3 with Zend Opcache at the same time to cache PHP files? I know that both do almost the same job, but not sure if that would make any difference on the performance and speed.
I want to speed up my php page load so that visitors don't need to wait long.
any thoughts on that?
To answer your question: Yes, you should not run xcache and Zend Opcache at the same time. If you do, you'll get undefined behaviours, most notably "cannot redeclare class XYZ" fatal errors. That happened to me after a systems upgrade, where the packet maintainer activated Zend Opcache along the already existing xcache installation.
As for the matter of which of both to use for opcaching, that depends on your specific code - I'd recommend setting up a test environment and firing up the Apache Benchmark or a similar tool to check the answer times.
On a default wordpress installation, I was able to get a speedup (uncached vs xcache) of about 5-7x, which is quite significant. If you really need more, you'll need to check out the other possibilities already mentioned in the comments like