httphaskellhttpshttp-conduitnetwork-conduit

Haskell simpleHTTP of Network.HTTP.Conduit performing slowly for get requests

In my haskell code I've imported Network.HTTP.Conduit as

import Network.HTTP.Conduit

In my main function I make a simple GET request using simpleHTTP

main = do
     response <- simpleHttp "https://github.com/trending?l=ruby"
     return ()

It took 6 minutes 42 secs to complete 100 api requests

time for NUM in `seq 1 1 100`; do ./Testhttp; done
real    6m42.839s
user    0m12.115s
sys 0m2.652s

whereas the ruby alternative just takes 153 secs for 100 api calls using Net::HTTP.get(URI.parse("https://github.com/trending?l=ruby"))

Am I doing something wrong in my haskell code? What are the performant and efficient alternatives to simpleHTTP?

Solution

  • The documentation for simpleHttp says:

    Note: This function creates a new Manager. It should be avoided in production code.

    Your code creates a new manager for each request. If you change it to reuse a single manager, it could be a lot faster. You can use newManager to create a manager. For example:

    import Network.HTTP.Conduit
main = do
    request <- parseUrl "https://github.com/trending?l=ruby"
    manager <- newManager conduitManagerSettings
    _response <- httpLbs request manager
    return ()