Aside from the image not being properly represented in something like an image editor, if my goal is to simply compress the pixels, and then retrieve them by decoding and reading them as I wrote them, technically it wouldn't matter whether they are signed or unsigned, little endian or bit endian, right? In the end it's just a bunch of bits, right? And signed or unsigned or little endian or big endian shouldn't affect the encoding and decoding, I should get back the exact values I put in, right?
Yes. It is just a bunch of bits that would be losslessly reproduced upon decompression. What the bits mean to you is irrelevant.