Lets start off that this is my first python program, so I will have some misconceptions on how parts of the code work.
Data is extracted from a 8-bit memory, this data is saved in a HDF5 file and I need to work with this HDF5 data. The dictionary I want to work with of this file has 3 keys with each key having multiple values. One of the keys is a counter and when it reached 256, it will reset to 0. This is expected due to the 8-bit memory.
So my challenge is to read the data of this key and when it overflows, then instead of resetting, counting up from the previous value. this is needed to make continues plots with the counter on the x-axis.
My issues starts with how to read out the values of a specific key one-by-one and, when needed, changing it. For the semi-working code(that resets the counter to 0, but does plot it) I used zip() to unpack the dict to a tuple(?). But after this i'm stuck again.
Here is the whole code so far.
import h5py
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path
measdatapath = Path(r'C:\Users\...')
title = 'Best_test_data_ever.hdf5'
plotpath = measdatapath / title
data = h5py.File(plotpath, 'r')
list(data.keys())
relevant_data = data['HDF5Data'][:]
# variable z is the counter
x, y, z = zip(*relevent_data)
fig, axs = plt.subplots(2,1, sharex=True)
axs[0].plot(z,x)
axs[0].set_ylabel("Awesome line")
axs[0].xaxis.set_label_position('top')
axs[0].set_title(title)
axs[1].plot(z,y)
axs[1].set_ylabel("Almost as awesome line")
axs[1].set_xlabel("Broken counter")
plt.show()
to summarize my challenges:
Note: above is scrambled code from different sources, I have a limited idea of what I am doing, but eager to learn.
so far I have tried to look up online how to solve this with a tuple or dict, but it didn't help.
Edit: The counting goes from 0 to 255 and then resets to 0. Below the current counting I have from the data:
(3, 7, 10, 14, 17, 21, 24, 28, 31, 34, 39, 42, 45, 48, 51, 54, 57, 61, 64, 67, 70, 73, 77, 80, 84, 88, 91, 94, 98, 100, 104, 107, 111, 114, 117, 121, 124, 127, 131, 133, 137, 140, 143, 146, 150, 153, 157, 160, 164, 167, 170, 173, 176, 179, 183, 186, 189, 191, 195, 198, 201, 203, 207, 210, 213, 217, 220, 223, 227, 230, 233, 236, 240, 243, 246, 249, 253, 1, 4, 8, 11, 15, 18, 22, 25, 28, 32, 35, 39, 42, 46, 49, 52, 55, 58, 61, 65, 68, 72, 75)
There are probably many ways of fixing up that counter. Here is one:
We start by ensuring that we run on numpy arrays. See my last side-note at the bottom about this.
z = np.asarray(z)
We then detect that a wrap-around happened:
wraparound = z[1:] < z[:-1]
This gives us an array like [false, false, ..., true, false, ..., true, false, ...]. We convert this to an integer array [0, 0, ..., 1, 0, ..., 1, 0, ...].
wraparound = wraparound.astype(int)
We can then use np.cumsum to accumulate how often a wrap-around happened up to the given index. This gives us an array [0, 0, ..., 1, 1, ... 2, 2, ...].
wraparound = wraparound.cumsum()
Now we just need to multiply with 256 and add it to the counter. Note that the wraparound array has one fewer entries than the base counter.
counter = z.astype(int)
counter[1:] += wraparound
To write it in as few lines as possible:
z = np.asarray(z)
counter = z.astype(int)
counter[1:] += (z[1:] < z[:-1]).astype(int).cumsum() * 256
Side note: x, y, z = zip(*relevent_data) is very inefficient since it replaces all numpy arrays with basic Python types. Write this instead to get transposed array views: x, y, z = relevent_data.T