pythonfitsastropypyfits

astropy.io.fits - HIERARCH keywords don't work with CONTINUE cards: Bug or "Feature" of the FITS standard?


The astropy.io.fits manual states, that we can use header keywords longer than 8-characters. In this case HIERARCH cards will be created. The manual also states, that if we want to store keyword-value pairs longer than 80-characters, continue cards will automatically be created.

However, in practice it seems that both definitions work only mutually exclusive, i.e. we can not create a FITS file containing a keyword value pair, where the keyword is longer than 8-characters (i.e. a HIERARCH keyword) and the value is a very long string.

An example:

from astropy.io import fits

header1 = fits.Header()
header2 = fits.Header()
header3 = fits.Header()

header1['TEST'] = 'superlongstring'*10
header2['TEST TEST'] = 'superlongstring'
header3['TEST TEST'] = 'superlongstring'*10

Here header1 and header2 will be correct, but when calling repr(header3) or attempting to save a FITS file with such a header, the error ValueError: The keyword TEST TEST with its value is too long is raised.

Is this an "unintended feature" of the FITS standard, i.e. can HIERARCH keywords not be continued with CONTINUE cards or might this be simply a bug of astropy.io.fits?


Solution

  • I wrote this in the issue opened by the OP, but I'm copying a version of it here too as a possible answer:

    The issue here is that the CONTINUE convention is strictly limited to extending the value of cards with string values (i.e. it does not apply to cards with different types of values). However, formally speaking, cards using the HIERARCH convention do not have any value at all, and so the CONTINUE convention does not apply to it, unless one were to implicitly support the CONTINUE convention within parsing of cards using the HIERARCH convention. Because there are ambiguities in how to do this (for example what is the maximum length of a keyword?) and because the HIERARCH convention does not explicitly allow for this interpretation (it gives a specific set of guidelines for how to interpret the contents of a HIERARCH card) FITS readers should refrain from adding any implicit support for some undocumented interpretation of HIERARCH cards (thus insituting yet another implicit convention that would not be supported by most readers/writers).

    What one could do is lobby the authors of the ESO HIERARCH convention to update their convention for explicit mutual compatibility with the CONTINUE convention, at which time this should be allowed in any FITS readers that support both conventions. But in the meantime the issue is too fraught with difficulties, even if it seems "obvious" that they could work together.