character-encodingrustiso-8859-1

What are the options to convert ISO-8859-1 / Latin-1 to a String (UTF-8)?


I scanned the Rust documentation for some way to convert between character encodings but did not find anything. Did I miss something?

Is it supported (directly or indirectly) by the Rust language and its standard libraries or even planned to be in the near future?

As one of the answers suggested that there is an easy solution because u8 can be cast to (Unicode) chars. With Unicode being a superset of the codepoints in ISO-8859-1, thats a 1:1 mapping which encodes to multiple bytes in UTF-8 which is the internal encoding of Strings in Rust.

fn main() {
    println!("{}", 196u8 as char);
    println!("{}", (196u8 as char) as u8);
    println!("{}", 'Ä' as u8);
    println!("{:?}", 'Ä'.to_string().as_bytes());
    println!("{:?}", "Ä".as_bytes());
    println!("{}",'Ä' == 196u8 as char);
}

gives:

Ä
196
196
[195, 132]
[195, 132]
true

Which I had not even considered to work!


Solution

  • Strings in Rust are unicode (UTF-8), and unicode codepoints are a superset of iso-8859-1 characters. This specific conversion is actually trivial.

    fn latin1_to_string(s: &[u8]) -> String {
        s.iter().map(|&c| c as char).collect()
    }
    

    We interpret each byte as a unicode codepoint and then build a String from these codepoints.