pythonimage-processingpython-imaging-librarypgm

Is there a proper way to convert common picture file extensions into a .PGM "P2" using PIL or cv2?


Edit: Problem solved and code updated.

I apologize in advance for the long post. I wanted to bring as much as I could to the table. My question consists of two parts.

Background: I was in need of a simple Python script that would convert common picture file extensions into a .PGM ASCII file. I had no issues coming up with a naive solution as PGM seems pretty straight forward.

# convert-to-pgm.py is a script for converting image types supported by PIL into their .pgm 
# ascii counterparts, as well as resizing the image to have a width of 909 and keeping the 
# aspect ratio. Its main purpose will be to feed NOAA style images into an APT-encoder 
# program.

from PIL import Image, ImageOps, ImageEnhance
import numpy as np

# Open image, convert to greyscale, check width and resize if necessary
im = Image.open(r"pics/NEKO.JPG").convert("L")

image_array = np.array(im)
print(f"Original 2D Picture Array:\n{image_array}")  # data is stored differently depending on 
                                                     # im.mode (RGB vs L vs P)
image_width, image_height = im.size
print(f"Size: {im.size}")      # Mode: {im.mode}")
# im.show()
if image_width != 909:
    print("Resizing to width of 909 keeping aspect ratio...")
    new_width = 909
    ratio = (new_width / float(image_width))
    new_height = int((float(image_height) * float(ratio)))
    im = im.resize((new_width, new_height))
    print(f"New Size: {im.size}")
# im.show()

# Save image data in a numpy array and make it 1D.
image_array1 = np.array(im).ravel()
print(f"Picture Array: {image_array1}")

# create file w .pgm ext to store data in, first 4 lines are: pgm type, comment, image size, 
# maxVal (=white, 0=black)
file = open("output.pgm", "w+")
file.write("P2\n# Created by convert-to-pgm.py \n%d %d\n255\n" % im.size)

# Storing greyscale data in file with \n delimiter
for number in image_array1:
    # file.write(str(image_array1[number]) + '\n')   #### This was the culprit of the hindered image quality...changed to line below. Thanks to Mark in comments.
    file.write(str(number) + '\n')
file.close()
im = im.save(r"pics/NEKO-greyscale.jpg")

# Strings to replace the newline characters
WINDOWS_LINE_ENDING = b'\r\n'
UNIX_LINE_ENDING = b'\n'

with open('output.pgm', 'rb') as open_file:
    content = open_file.read()

content = content.replace(WINDOWS_LINE_ENDING, UNIX_LINE_ENDING)

with open('output.pgm', 'wb') as open_file:
    open_file.write(content)
open_file.close()

This produces a .PGM file that, when opened with a text editor, looks similar to the same image that was exported as a .PGM using GIMP (My prior solution was to use the GIMP export tool to manually convert the pictures and I couldn't find any other converters that supported the "P2" format). However, the quality of the resulting picture is severely diminished compared to what is produced using the GIMP export tool. I have tried a few methods of image enhancement (brightness, equalize, posterize, autocontrast, etc.) to get a better result, but none have been entirely successful. So my first question: what can I do differently to obtain a result that looks more like what GIMP produces? I am not looking for perfection, just a little clarity and a learning experience. How can I automatically adjust {insert whatever} for the best picture?

Below is the .PGM image produced by my version compared GIMP's version, open in a text editor, using the same input .jpg

My version vs. GIMP's version:

text editor comparisons

Below are comparisons of adding various enhancements before creating the .pgm file compared to the original .jpg and the original .jpg converted as a greyscale ("L"). All photos are opened through GIMP.

Original .jpg

original .jpg image, before conversion

Greyscale .jpg, after .convert("L") command

greyscale .jpg image, before conversion

**This is ideally what I want my .PGM to look like. Why is the numpy array data close, yet different than the data in the GIMP .PGM file, even though the produced greyscale image looks identical to what GIMP produces? Answer: Because it wasn't saving the correct data. :D

GIMP's Resulting .PGM

GIMP's result of the .PGM

My Resulting .PGM

resulting original .pgm

My Resulting .PGM with lower brightness, with Brightness.enhance(0.5)

.PGM half as bright

Resulting .PGM with posterize, ImageOps.posterize(im, 4)

.PGM with posterize of 4

SECOND PROBLEM: My last issue comes when viewing the .PGM picture using various PGM viewers, such as these online tools (here and here). The .PGM file is not viewable through one of the above links, but works "fine" when viewing with the other link or with GIMP. Likewise, the .PGM file I produce with my script is also not currently compatible with the program that I intend to use it for. This is most important to me, since its purpose is to feed the properly formatted PGM image into the program. I'm certain that something in the first four lines of the .PGM file is altering the program's ability to sense that it is indeed a PGM, and I'm pretty sure that it's something trivial, since some other viewers are also not capable of reading my PGM. So my main question is: Is there a proper way to do this conversion or, with the proper adjustments, is my script suitable? Am I missing something entirely obvious? I have minimal knowledge on image processing.

GitHub link to the program that I'm feeding the .PGM images into: here

More info on this particular issue: The program throws a fault when ran with one of my .PGM images, but works perfectly with the .PGM images produced with GIMP. The program is in C++ and the line "ASSERT(buf[2] == '\n')" returns the error, implying that my .PGM file is not in the correct format. If I comment this line out and recompile, another "ASSERT(width == 909)..." throws an error, implying that my .PGM does not have a width of 909 pixels. If I comment this line out as well and recompile, I am left with the infamous "segmentation fault (core dumped)." I compiled this on Windows, with cygwin64. Everything seems to be in place, so the program is having trouble reading the contents of the file (or understanding '\n'?). How could this be if both my version and GIMP's version are essentially identical in format, when viewed with a text editor?

Terminal output:

output from program in the terminal

Thanks to all for the help, any and all insight/criticism is acceptable.


Solution

  • The first part of my question was answered in the comments, it was a silly mistake on my end as I'm still learning syntax. The above code now works as intended.

    I was able to do a little more research on the second part of my problems and I noticed something very important, and also feel quite silly for missing it yesterday.

    Windows vs Unix encoding

    So of course the reason why my program was having a problem reading the '\n' character was simply because Windows encodes newline characters as CRLF aka '\r\n' as opposed to the Unix way of LF aka '\n'. So in my script at the very end I just add the simple code [taken from here]:

    # replacement strings
    WINDOWS_LINE_ENDING = b'\r\n'
    UNIX_LINE_ENDING = b'\n'
    
    
    with open('output.pgm', 'rb') as open_file:
        content = open_file.read()
    
    content = content.replace(WINDOWS_LINE_ENDING, UNIX_LINE_ENDING)
    
    with open('output.pgm', 'wb') as open_file:
        open_file.write(content)
    

    Now, regardless on whether the text file is encoded with CRLF or LF, the script will work properly.