My Question/ Expectation is to find a possible point or statment that could cause such problems. If you have experienced something similar in JS, where it does go through the whole Function but only calculations one part, and have a solution that would still help?
I'm currently fixing some stuff in my code of my web worker and broke something and its not throwing an error, what's weird is that it only does a bit of the calculation (only the stuff for CandleSeriesArray). I have no clue what to do since logging the data that's send is correct which is also confirmed by the CandleSeriesArray:
First some information on how the code works the part with the calculation is further down: In Worker file - Worker receive and send
let testCounter = 0;
let counter2 = 0;
let preparationCounter = 0;
//worker.postMessage({chunk: chunk, index: i, preparationChunk: preparationChunk, changer: changer});
self.onmessage = function (e) {
const { chunk, index, preparationChunk, changer } = e.data;
preparationChunk.forEach(line => {
//var message = line;//JSON.parse(line);
if (line.m === false) {
processCandleData(line.E, line, false, changer)
};
preparationCounter++;
});
// Iterate over each line and parse as JSON
chunk.forEach(line => {
//var message = line;//JSON.parse(line);
if (line.m === false) {
calculateKline = processCandleData(line.E, line, true, changer)
testCounter++;
};
counter2++;
});
// Send the processed result back to the main script
self.postMessage({ index: index, result: calculateKline, counter: counter2, preparationCounter: preparationCounter , testCounter: testCounter });
};
First the data received:
worker.postMessage({chunk: chunk, index: i, preparationChunk: preparationChunk, changer: changer});
chunk is an array that consist of many lines in this format, which is the same for the preparationChunk. This is then send to the function processCandleData(line.E, line, false, changer), line is the placeholder for both data Chunks.
{m: false, E: 1716823048658, k: {…}}
1. E: 1716823048658
2. k:
1. T: 1716823079999
2. c: "69968"
3. i: "1m"
4. o: 69968
5. s: "BTCFUSDT"
6. t: 1716823020000
3. m: false
index is the number of the worker, changer has the value 5 and is needed for the calculation of the SMA value in the processCandleData function under the comment: //Update the data for the SMA series
the following part:
self.postMessage({ index: index, result: calculateKline, counter: counter2, preparationCounter: preparationCounter , testCounter: testCounter
sends it back to the main script result: calculateKline looks in the console like this (result after two worker were done):
{CandleSeriesArray: Array(26034), SmaSeriesArray: Array(0), CloseValueSeriesArray: Array(0)}
{CandleSeriesArray: Array(26842), SmaSeriesArray: Array(0), CloseValueSeriesArray: Array(1)}
This shows me that the arrays for SmaSeriesArray and CloseValueSeriesArray are missing. What it should actual look like is something like this: (numbers are made up)
{CandleSeriesArray: Array(26034), SmaSeriesArray: Array(10304), CloseValueSeriesArray: Array(234)}
{CandleSeriesArray: Array(26842), SmaSeriesArray: Array(10604), CloseValueSeriesArray: Array(236)}
this is done by the processCandleData return which is then but in the result value of the send back message of the worker
First I wanted to check one of the 2 call function that don't work. The first part works fine:
if (ChunkProcessing === true) { // === true not needed but it's there for clarity
const CandleSeries = createCandleSeries(candlestick);
CandleSeriesArray.push(CandleSeries);
}
But this part if (PreviousCandlestick && PreviousCandlestick.T < candlestick.t) (located at around line 39) doesn't work but it must be true since PreviousCandlestick is defined and T is always a full minute timestamp when t is higher then that minute then the statement is true. My data has enough for multiple hours so that means that it must at one point be higher then a previous full minute.
combinedMessageArrayLength: (5)
[26034, 37019, 36698, 41044, 37896]
Calculation
// I do not know if those are all arrays in the function, but in my code all arrays are defined
let CandleSeriesArray = [];
let SmaSeriesArray = [];
let CloseValueSeriesArray = [];
let MessageArray = [];
let LastCandlestickArray = [];
let closeValuesArray = [];
//-- CALCULATION --//
function createCandleSeries(candlestick) {
return {
time: candlestick.t / 1000,
open: candlestick.o,
high: candlestick.h,
low: candlestick.l,
close: candlestick.c
};};
function createCloseValueSeries(LastCandlestick, CloseValue) {
return {
time: LastCandlestick.t / 1000,
value: CloseValue
};};
function createSmaSeries(LastCandlestick, SMA, colorSMA) {
return {
time: LastCandlestick.t / 1000,
value: SMA,
color: colorSMA
};};
function processCandleData(ActualTimestamp, message, ChunkProcessing, changer) {
var candlestick = message.k
if (ChunkProcessing === true) { // === true not needed but it's there for clarity
const CandleSeries = createCandleSeries(candlestick);
CandleSeriesArray.push(CandleSeries);
}
//return processedCandleData = processCandleData(klineData)
//CLOSEVALUE OF CANDLESTICK
MessageArray.push(candlestick)
let PreviousCandlestick = undefined;
if (MessageArray.length >= 2) {
PreviousCandlestick = MessageArray[MessageArray.length - 2]
};
if (PreviousCandlestick && PreviousCandlestick.T < candlestick.t) {
LastCandlestickArray.push(PreviousCandlestick)
LastCandlestick = LastCandlestickArray[LastCandlestickArray.length - 1];
CloseValue = LastCandlestick.c;
closeValuesArray.push(CloseValue);
if (ChunkProcessing === true) { // === true not needed but it's there for clarity
const CloseValueSeries = createCloseValueSeries(LastCandlestick, CloseValue);
CloseValueSeriesArray.push(CloseValueSeries);
}
};
//Update the data for the SMA series
if (closeValuesArray.length >= changer) {
// CALCULATION ON EVERY CLOSE //
if (closeValuesArray.length >= changer && !cal_on_every_tick) {
lastChangerValues = closeValuesArray.slice(-changer);
sumClose = lastChangerValues.reduce((a, b) => +a + +b);
SMA = sumClose / changer;
colorSMA = (CloseValue >= SMA) ? '#0000FF' : '#FFA500';
if (ChunkProcessing === true) { // === true not needed but it's there for clarity
const SmaSeries = createSmaSeries(LastCandlestick, SMA, colorSMA);
SmaSeriesArray.push(SmaSeries);
};
};
};
return {
CandleSeriesArray,
SmaSeriesArray,
CloseValueSeriesArray,
};
}
So I figured it out! I didn't realize when I checked the data since I only saw a minute change but didn't pay closer attention to it.
E: 1716823048658, T: 1716823079999, t: 1716823020000
E: 1716823079998, T: **1716823139999**, t: 1716823020000
E: 1716823080001, T: 1716823139999, t: **1716823080000**
The part you mentioned checks if the previous T value is lower than the current t which shows a new minute has started (in one line T is always > than t). since the data creation code made the T value of the previous candle (second one) the same as the T value of the actual minute change (third one), so 60000 higher than it should actual be. The t value could never be higher than T. This resulted in the third function also not getting data since it is based on the second function. this is how the data should look like:
E: 1716823048658, T: 1716823079999, t: 1716823020000
E: 1716823079998, T: **171682079999,** t: 1716823020000
E: 1716823080001, T: 1716823139999, t: 1716823080000
Conclusion: always check data thoroughly! And thx for the help so far