I'm working on a bigger project rewrite, with quite a big codebase already written in neko. One of the aspects of the project is a data scraper which would (during peak hours) have 100+ connections open to a WebSockets server. Originally, this was done with lots of nodejs processes running, using a WebSockets npm package. The problem was that this was somewhat unreliable, and would slow down the machine running these processes quite a lot. I hoped to solve this with Threads running in a single neko process.
But, I ran into a problem where I didn't expect it – the very awkward support (or lack thereof) of SSL / TLS in haxe. As I understand, the only native OpenSSL wrapper available is the hxssl
haxelib. I installed it, but it didn't work with the WebSockets still, so I traced the problem to a simpler case – just a single HTTPS connection, like so:
import haxe.Http;
class Main {
public static function main(){
var http = new Http("https://www.facebook.com/");
http.certFolder = 'certs';
http.certFile = 'certs/ca-certificates.crt';
http.setHeader("Accept", "text/html,application/xhtml+xml,application/xml");
http.setHeader("Accept-Language", "en-US");
http.setHeader("Cache-Control", "max-age=0");
http.setHeader("Connection", "close");
http.setHeader("DNT", "1");
http.setHeader("Upgrade-Insecure-Requests", "1");
http.setHeader("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36");
http.onData = function(data:String){
Sys.println("Data: " + data.substr(0, 50) + " ...");
}
http.onError = function(msg:String){
Sys.println("Error: " + msg);
}
http.onStatus = function(status:Int){
Sys.println("Status: " + status);
}
http.request(false);
}
}
The problem is that sometimes the output of this is simply:
Status: 200
Error: Custom((1) : An unknown error has occurred.)
And the worst part is the randomness with which this happens. Sometimes it happens a number of times in a row, even if I don't rebuild the project. I'm running this on an OS X machine at the moment.
The certs
folder is filled with certificates copied from the certs on an up-to-date Ubuntu server. I've tried without the certFolder
and certFile
lines, with pretty much the same results, however.
Any ideas about what could cause this? Writing a better wrapper / native implementation of OpenSSL is probably out of question, I'm somewhat pressed for time. I tried a cpp build of the above, which failed spectacularly with Sockets code, I'm not sure I want to go down that road either.
Perhaps you can try the RC for the upcoming 3.3 release, it has built-in Neko/Hxcpp support for SSL/TLS.