node.jsexpressgraphqllong-polling

Alternative to GraphQL long polling on an Express server for a large request?


Objective

I need to show a big table of data in my React web app frontend.

My backend is an Express server with a GraphQL layer and a few "normal" endpoints.

My server gets data from various sources, including an external API, which is the data source for my current task.

My server has a database that I can use freely. I cannot directly access the external API from my front end.

The data all comes from the external API I mentioned. In fact, it comes from multiple similar calls to the same endpoint with many different IDs. Each of those individual calls takes a while to return but doesn't risk timing out.

Current Solution

My naive implementation: I do one GraphQL query in which the resolver does all the API calls to the external service in parallel. It waits on them all to complete using Promise.all(). It then returns a big array containing all the data I need to my server. My server then returns that data to me.

Problem With Current Solution

Unfortunately, this sometimes leaves my frontend hanging for too long and it times out (takes longer than 2 minutes).

Proposed Solution

Is there a better way than manually implementing long polling in GraphQL? This is my main plan for a solution at the moment:

Question

Is there a better alternative to GraphQL long polling on an Express server for this large request that I have to do?

An alternative that comes to mind is to not use GraphQL and instead use a standard HTTP endpoint, but I'm not sure that really makes much difference.

I also see that HTTP/2 has multiplexing which could be relevant. My server currently runs HTTP/1.1 and upgrading is something of an unknown to me.

I see here that Keep-Alive, which sounds like it could be relevant, is unusable in Safari which is bad as many of my users use Safari to access the frontend.

I can't use WebSockets because of technical restraints. I don't want to set a ridiculously long timeout on my client either (and I'm not sure if it's possible)


Solution

  • I discovered that GraphQL has polling built in https://www.apollographql.com/docs/react/data/queries/#polling

    In the end, I made a REST polling system.