pythonsizepickle

How to check size of a pickle or number of contents it has


I made a pickle long time ago and want to use what's inside it, but I have to make sure I have the exact amount [8752 pickled images] inside it. I tried usual python function like len()but it failed. But when I try printing one of it's content like an array, it does output an array that looks like an array of images and that apparently cannot be converted into visible images (like .jpeg) again.

How do I make sure I have all images inside?

I tried printing the images per array number and it outputs an array of arrays, I tried checking usual way of checking size in python with 'len' but it outputs 1080 instead of expected 8752 image content.

enter image description here

Edit 1:

The codes for loading pickle

#load pickle
with open("rivergate_image.pkl", "rb") as f:

    river_images = pickle.load(f)

print(river_images[0])
print(len(river_images))

edit 2:

### print(river_images) output
[[[ 27 147 145]
  [ 30 150 148]
  [ 28 148 146]
  ...
  [ 61 178 169]
  [ 63 177 169]
  [ 65 176 167]]

 [[ 24 144 142]
  [ 24 144 142]
  [ 24 144 142]
  ...
  [ 59 179 170]
  [ 63 177 169]
  [ 65 176 169]]

 [[ 32 152 150]
...
  ...
  [ 37 142 145]
  [ 36 141 144]
  [ 35 140 143]]]
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

Edit 3: output of

print(len(river_images))

are

1080

output of

print(type(river_images))

are

<class 'numpy.ndarray'>

output of

print(river_images.shape)

are

(1080, 1420, 3)

Solution

  • Looks like you dumped the images separately into the file, as NumPy arrays (1080x1420 pixels of red+green+blue value).

    Try loading and counting until the end of the file. Example:

    import pickle
    
    # Create test file
    with open('test.pkl', 'wb') as f:
      for thing in 'foobar':
        pickle.dump(thing, f)
    
    # Load/count
    with open('test.pkl', 'rb') as f:
      n = 0
      try:
        while True:
          thing = pickle.load(f)
          n += 1
          print(repr(thing))
      except EOFError:
        print(n, 'things')
    

    Output (Attempt This Online!):

    'f'
    'o'
    'o'
    'b'
    'a'
    'r'
    6 things