libmemcached - memcached_mget seems to block

I have a single memcached server. I use the libmemcached C api to interface with it. I am using non blocking mode and no reply mode (behavior flags).

I am also using libmemcached inside an nginx C module if that helps.

The problem is that memcached_mget seems to block, ie for a server which has a latency of about 40ms, memcached_mget takes 40ms to complete. This is not exactly async.

Here is the code that I am using:

const char* localKeys[2] = {"key1", "key2"};
size_t k_length[2] = {4, 4};

gettimeofday(&t1, NULL);
ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, "no block: %d ", memcached_behavior_get(memc, MEMCACHED_BEHAVIOR_NO_BLOCK));
ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, "no reply: %d ", memcached_behavior_get(memc, MEMCACHED_BEHAVIOR_NOREPLY));
retFt = memcached_mget(memc, localKeys, k_length, 2);
gettimeofday(&t2, NULL);

ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, "memcached_mget duration (microseconds): %d, result : %d", t2.tv_usec - t1.tv_usec, retFt);

retFt = memcached_fetch_execute(memc, fp, &callback_struct, 1);

I am using ASCII protocol. Tried using binary also, but it didn't help. I also tried memcached_mget_execute. I am simulating the latency using tc, like so :

tc qdisc add dev lo root netem delay 20ms

libmemcached's API is always blocking. The "nonblocking" mode just sets that flag on the socket. If you use that flag, libmemcached does its own blocking by calling poll().