I am trying to send images captured from a canvas to my NodeJS backend server using the WebRTC data channel. That is I am trying to make my server a peer. But for some reason, I am unable to establish a connection.
async function initChannel()
{
const offer = await peer.createOffer();
await peer.setLocalDescription(offer);
const response = await fetch("/connect", {
headers: {
'Content-Type': 'application/json',
},
method: 'post',
body: JSON.stringify({ sdp: offer, id: Math.random() })
}).then((res) => res.json());
peer.setRemoteDescription(response.sdp);
const imageChannel = peer.createDataChannel("imageChannel", { ordered: false, maxPacketLifeTime: 100 });
peer.addEventListener("icecandidate", console.log);
peer.addEventListener("icegatheringstatechange",console.log);
// drawCanvas function draws images got from the server.
imageChannel.addEventListener("message", message => drawCanvas(remoteCanvasCtx, message.data, imageChannel));
// captureImage function captures and sends image to server using imageChannel.send()
imageChannel.addEventListener("open", () => captureImage(recordCanvasCtx, recordCanvas, imageChannel));
}
const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
initChannel();
Here both captureImage
and drawCanvas
are not being invoked.
import webrtc from "wrtc"; // The wrtc module ( npm i wrtc )
function handleChannel(channel)
{
console.log(channel.label); // This function is not being called.
}
app.use(express.static(resolve(__dirname, "public")))
.use(bodyParser.json())
.use(bodyParser.urlencoded({ extended: true }));
app.post("/connect", async ({ body }, res) =>
{
console.log("Connecting to client...");
let answer, id = body.id;
const peer = new webrtc.RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
await peer.setLocalDescription(answer = await peer.createAnswer());
peer.addEventListener("datachannel",handleChannel)
return res.json({ sdp: answer });
});
app.listen(process.env.PORT || 2000);
Here the post request is handled fine but handleChannel
is never called.
When I run this I don't get any errors but when I check the connection status it shows "new" forever. I console logged remote and local description and they seem to be all set. What am I doing wrong here?
I am pretty new to WebRTC and I am not even sure if this is the correct approach to continuously send images (frames of user's webcam feed) to and back from the server, if anyone can tell me a better way please do.
And one more thing, how can I send image blobs ( got from canvas.toBlob()
) via the data channel with low latency.
I finally figured this out with the help of a friend of mine. The problem was that I have to create DataChannel before calling peer.createOffer()
. peer.onnegotiationneeded
callback is only called once the a channel is created. Usually this happens when you create a media channel ( either audio or video ) by passing a stream to WebRTC, but here since I am not using them I have to to this this way.
const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
const imageChannel = peer.createDataChannel("imageChannel");
imageChannel.onmessage = ({ data }) =>
{
// Do something with received data.
};
imageChannel.onopen = () => imageChannel.send(imageData);// Data channel opened, start sending data.
peer.onnegotiationneeded = initChannel
async function initChannel()
{
const offer = await peer.createOffer();
await peer.setLocalDescription(offer);
// Send offer and fetch answer from the server
const { sdp } = await fetch("/connect", {
headers: {
"Content-Type": "application/json",
},
method: "post",
body: JSON.stringify({ sdp: peer.localDescription }),
})
.then(res => res.json());
peer.setRemoteDescription(new RTCSessionDescription(sdp));
}
Receive offer from client sent via post request. Create an answer for it and send as response.
app.post('/connect', async ({ body }, res) =>
{
const peer = new webrtc.RTCPeerConnection({
iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
});
console.log('Connecting to client...');
peer.ondatachannel = handleChannel;
await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
await peer.setLocalDescription(await peer.createAnswer());
return res.json({ sdp: peer.localDescription });
});
The function to handle data channel.
/**
* This function is called once a data channel is ready.
*
* @param {{ type: 'datachannel', channel: RTCDataChannel }} event
*/
function handleChannel({ channel })
{
channel.addEventListener("message", {data} =>
{
// Do something with data received from client.
});
// Can use the channel to send data to client.
channel.send("Hi from server");
}
So here is what happens :
onnegotiationneeded
callback is called.ondatachannel
callback gets called on the server and the client.I have used post request here to exchange offer and answer but it should be fairly easy to do the same using Web Socket if that is what you prefer.