asciidelimitercontrol-characters

Are most of the ASCII control characters obsolete?


Most of the ASCII codes under \x20 appear to be entirely obsolete. Are they used at all today? Can they be considered "up for grabs", or is it best to avoid them?

I need a delimiter for grouping "lines" together and it would sure be nice to co-opt one of these for that purpose.

From man ascii:

    Oct   Dec   Hex   Char                        
    ----------------------------------------------
    000   0     00    NUL '\0'                    
    001   1     01    SOH (start of heading)      
    002   2     02    STX (start of text)         
    003   3     03    ETX (end of text)           
    004   4     04    EOT (end of transmission)   
    005   5     05    ENQ (enquiry)               
    006   6     06    ACK (acknowledge)           
    007   7     07    BEL '\a' (bell)             
    010   8     08    BS  '\b' (backspace)        
    011   9     09    HT  '\t' (horizontal tab)   
    012   10    0A    LF  '\n' (new line)         
    013   11    0B    VT  '\v' (vertical tab)     
    014   12    0C    FF  '\f' (form feed)        
    015   13    0D    CR  '\r' (carriage ret)     
    016   14    0E    SO  (shift out)             
    017   15    0F    SI  (shift in)              
    020   16    10    DLE (data link escape)      
    021   17    11    DC1 (device control 1)      
    022   18    12    DC2 (device control 2)      
    023   19    13    DC3 (device control 3)      
    024   20    14    DC4 (device control 4)      
    025   21    15    NAK (negative ack.)         
    026   22    16    SYN (synchronous idle)      
    027   23    17    ETB (end of trans. blk)     
    030   24    18    CAN (cancel)                
    031   25    19    EM  (end of medium)         
    032   26    1A    SUB (substitute)            
    033   27    1B    ESC (escape)                
    034   28    1C    FS  (file separator)        
    035   29    1D    GS  (group separator)       
    036   30    1E    RS  (record separator)      
    037   31    1F    US  (unit separator)        
    040   32    20    SPACE                       

Solution

  • First the easy part: There are no network transmission concerns in most modern systems. Current protocols handle almost any data - whether 7-bit ASCII, 8-bit ASCII, Unicode characters, image data or compiled programs - as binary data. That has not always been the case. Many older systems had issues transferring control codes and other "unprintable" characters and especially problems with 8-bit data. But those days are, fortunately, behind us. The one big exception is if you want to be able to copy/paste data via an HTML form - for that you want to leave out all control codes and other funny stuff.

    You can, of course, make the format anything you like. However, some characters are still used pretty frequently:

    000   0     00    NUL '\0' - does "nothing" but is hard for some text editors to handle
    003   3     03    ETX (end of text) - Control-C - "break" in a lot of systems
    007   7     07    BEL '\a' (bell) - Still makes a bell sound.
    011   9     09    HT  '\t' (horizontal tab) - A lot of text editors and file formats use this to set a fixed number of spaces
    012   10    0A    LF  '\n' (new line) - like it says
    015   13    0D    CR  '\r' (carriage ret) - used instead of, or together with \n on many systems
    021   17    11    DC1 (device control 1) - Control-Q - Resume transmission - XON
    023   19    13    DC3 (device control 3) - Control-S - Pause transmission - XOFF
    033   27    1B    ESC (escape) - Used for PCL and other printer control codes and plenty of other things 
    

    Everything else is pretty much up for grabs. I would especially avoid NUL and XON/XOFF - they are sometimes hard to enter into a file - and BEL because typing a file with BEL can be noisy.

    If you have a truly binary format then you can do anything you want. But if you want to have a mostly-human-readable format then limiting the control codes is a good idea.