I am trying to consume data from this endpoint on lichess.org.
Here is a minimal working example of a React component consuming that stream of data. I'm using a library called can-ndjson-stream.
import ndjsonStream from "can-ndjson-stream"
import { useEffect } from "react"
function App() {
useEffect(() => {
fetch("https://lichess.org/api/tv/feed")
.then(res => ndjsonStream(res.body))
.then(stream => {
const streamReader = stream.getReader()
streamReader.read().then(async res => {
while (!res || !res.done) {
res = await streamReader.read()
console.log(res.value)
}
})
})
.catch(console.error)
}, [])
return <>Lorem Ipsum</>
}
export default App
However, if I try to write this same code and run it in Node like this:
import ndjsonStream from "can-ndjson-stream"
import fetch from "node-fetch"
fetch("https://lichess.org/api/tv/feed")
.then(res => ndjsonStream(res.body))
.then(stream => {
const streamReader = stream.getReader()
streamReader.read().then(async res => {
while (!res || !res.done) {
res = await streamReader.read()
console.log(res.value)
}
})
})
.catch(console.error)
I get this error:
ReferenceError: ReadableStream is not defined at ndjsonStream
So it seems like the res
from the fetch is null or undefined, but fetching other APIs works fine.
I also tried using axios instead of node-fetch like this:
import ndjsonStream from "can-ndjson-stream"
import axios from "axios"
axios
.get("https://lichess.org/api/tv/feed")
.then(res => ndjsonStream(res.data))
.then(stream => {
const streamReader = stream.getReader()
streamReader.read().then(async res => {
while (!res || !res.done) {
res = await streamReader.read()
console.log(res.value)
}
})
})
.catch(console.error)
But it just hangs and shows no output. Thanks to anyone that could shed some light on this or offer any alternative way to run this in Node.
Thanks to the comments from tromgy, I was able to make something that works. I went with the library hyperquest to help handle the request and piping of the stream. I also used the ndjson library.
Here is some working code:
hyperquest("https://lichess.org/api/tv/feed")
.pipe(ndjson.parse())
.on("data", console.log)
Note that you can manipulate the objects as they arrive with the second argument of on()
, like below.
...
.on("data", (obj) => {
foo(obj)
})