I am trying to create a TensorFlow Dataset from DICOM images using the tf.data API and tensorflow_io, and I want to perform some pre-processing using Hounsfield units from the images. The DICOM images have a shape of (512,512). I have extracted the PixelData from the image and want to convert it to a numpy array of appropriate shape using the following code:
image_bytes = tf.io.read_file(image_path)
PixelData = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData).numpy()
pixel_array = np.frombuffer(PixelData, dtype=tf.uint16)
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
This code should be equivalent to
Image = pydicom.dcmread(image_path)
pixel_array = Image.pixel_array
print(pixel_array)
and
Image = pydicom.dcmread(image_path)
PixelData = Image.PixelData
pixel_array = np.frombuffer(PixelData, dtype=np.uint16)
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
The DICOM tags are the same ones used by pydicom, and are given here. The PixelData should return the raw byte values of the DICOM image. I have confirmed through pydicom that the raw pixel data is stored as np.uint16 values. However, when I try to convert the byte data given by tensorflow into a numpy array using the np.frombuffer function, I get an error of buffer size not being divisible by the element length.
When I run the above scripts, these are the following output shapes
The pydicom examples provide identical outputs in both cases, however the tensorflow DICOM tag seems to provide a completely different result. Please find attached an example DICOM file here. Is there something wrong with the library or my implementation?
Edit: The DICOM images are actually signed 16 bit integers, not unsigned. Hence, the following three snippets of code produce the same output:
Direct pixel_array from pydicom
import pydicom
import numpy as np
dcm = pydicom.dcmread("ID_000012eaf.dcm")
print(dcm.pixel_array)
Converting PixelData to pixel_array manually
import pydicom
import numpy as np
dcm = pydicom.dcmread("ID_000012eaf.dcm")
PixelData = dcm.PixelData
pixel_array = np.frombuffer(PixelData, dtype=np.int16)
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
Directly obtaining pixel_array using tensorflow_io
import tensorflow as tf
import tensorflow_io as tfio
import numpy as np
image_bytes = tf.io.read_file("ID_000012eaf.dcm")
pixel_array = tfio.image.decode_dicom_image(image_bytes, on_error='lossy', scale='preserve', dtype=tf.float32).numpy()
pixel_array = pixel_array.astype('int16')
pixel_array /= 2.
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
However, this final snippet of code still doesn't work for some reason:
import tensorflow as tf
import tensorflow_io as tfio
import numpy as np
image_bytes = tf.io.read_file("ID_000012eaf.dcm")
PixelData = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData).numpy()
pixel_array = np.frombuffer(PixelData, dtype=np.int16)
print(pixel_array)
Edit 2: These two snippets of code should theoretically work, however they display errors that the length of the byte string is not divisible by the size of int16:
import tensorflow as tf
import tensorflow_io as tfio
import numpy as np
image_bytes = tf.io.read_file("ID_000012eaf.dcm")
PixelData = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData).numpy()
pixel_array = np.frombuffer(PixelData, dtype=np.int16)
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
and
import tensorflow as tf
import tensorflow_io as tfio
import numpy as np
image_bytes = tf.io.read_file("ID_000012eaf.dcm")
PixelData = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData)
pixel_array = tf.io.decode_raw(PixelData, tf.int16)
pixel_array = tf.reshape(pixel_array, (512,512))
print(pixel_array)
Edit 3: After getting the hint that the byte string provided by decode_dicom_data contains hexadecimal values, I found a way to convert my data into the desired pixel_array, but I'm curious why PixelData is stored this way:
import tensorflow as tf
import tensorflow_io as tfio
import numpy as np
image_bytes = tf.io.read_file("ID_000012eaf.dcm")
PixelData = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData).numpy()
pixel_array = np.zeros(262144, dtype=np.int16)
start,stop = 0,4
for i in range(262144):
pixel_array[i] = int(PixelData[start:stop], base=16)
start+=5
stop+=5
pixel_array = np.reshape(pixel_array, (512,512))
print(pixel_array)
PixelData from pydicom:
PixelData = b'0\xf80\xf80\xf80...'
PixelData from Tensorflow_io
PixelData = b'f830\\f830\\f830\\...'
Any suggestions for code refactoring and linting would be highly appreciated. I am extremely grateful to @ai2ys for helping me diagnose these issues.
The function tfio.image.decode_dicom_data decodes the tag information and not the pixel information.
To read the pixel data use tfio.image.decode_dicom_image instead.
import tensorflow_io as tfio
image_bytes = tf.io.read_file(image_path)
pixel_data = tfio.image.decode_dicom_image(
image_bytes,
dtype=tf.uint16)
# type conversion and reshaping is not required
# as can be checked with the print statement
print(pixel_data.dtype, pixel_data.shape)
# if required the pixel_data can be converted to a numpy array
# but calculations like scaling and offset correction can
# be done on tensors as well
pixel_data_nparray = pixel_data.numpy()
# reading tag information, e.g. rescale intercept and slope
intersept = tfio.image.decode_dicom_data(
image_bytes,
tfio.image.dicom_tags.RescaleIntercept)
slope = tfio.image.decode_dicom_data(
image_bytes,
tfio.image.dicom_tags.RescaleSlope)
print(intersept)
print(slope)
Please checkout the docs for further information:
Edit 2021-02-01 using the shared file:
It is also possible to read the pixel data using tfio.image.decode_dicom_data with passing tfio.image.dicom_tags.PixelData but the byte string getting returned has to be decoded.
data = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PixelData)
print(data)
Output (shortened):
tf.Tensor(b'f830\\f830\\f830\\f830\\ ...')
The hex value f830 interpreted as int16 is -2000.