rmatlab

Out of range values when reading a MATLAB file with R.matlab

I have dowloaded a MATLAB file with species names and their ecological group from https://ambi.azti.es/download/

In step 5 on this page the file library.mat can be downloaded (zipped). I then use the function readMat() from the R.matlab library to read this file:

library(R.matlab)    
ambi<-readMat("library.mat")

I get this 7 warnings:

Warning messages:

1: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
2: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
3: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
4: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
5: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
6: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw  
7: In convertUTF8(ary) :
out-of-range values treated as 0 in coercion to raw

I don't know what this means but I suspect it has something to do with encodings. I would be very thankful if someone can tell me what the reason is and how I can solve this problem. I don't have MATLAB. I am using R version 4.4.3 on Swedish Windows 11.

Solution

  • I looks like R.matlab is doing you a favour by omitting those characters. While I am not an SME, an internet search suggests those characters are often omitted from the affected species names so perhaps they are errors? At least there are only seven to deal with if you have to manually edit them.

    To examine the issue, you can use the rmatio package, as it will retain the non-UTF characters.

    library(rmatio)
library(stringi)

# Unzip .mat previously downloaded from
# https://azti.sharepoint.com/sites/Proyectos/AMBI/Documentos%20compartidos/Forms/AllItems.aspx?id=%2Fsites%2FProyectos%2FAMBI%2FDocumentos%20compartidos%2FLibrary%2D2024%2Ezip&parent=%2Fsites%2FProyectos%2FAMBI%2FDocumentos%20compartidos&p=true&ga=1
# Edit file path where necessary
unzip("data/AMBI/Library-2024.zip",
      files = "library.mat",
      exdir = "data/AMBI/")

ambi <- read.mat("data/AMBI/library.mat")

# Convert to df
df <- data.frame(name = unlist(ambi$specieslist$name),
                 group = unlist(ambi$specieslist$group),
                 reassign = unlist(ambi$specieslist$reassign))

# Return rows containing non-UTF
x <- df[which(stri_enc_mark(df$name) == "native"),]

x$name
# [1] "Congetia chesneyi\xfd"     "Erythrops go\xfdsi"        "Erythrops go\xfdsii"      
# [4] "Gymnonereis\xfdcrosslandi" "Hippolyte cura\xfdaoensis" "Neorhynchoplax\xfdsp."    
# [7] "Valencinia\xfdsp."

# Convert non-UTF chars
iconv(x$name, from = "macroman", to = "UTF-8")
# [1] "Congetia chesneyi˝"     "Erythrops go˝si"        "Erythrops go˝sii"      
# [4] "Gymnonereis˝crosslandi" "Hippolyte cura˝aoensis" "Neorhynchoplax˝sp."    
# [7] "Valencinia˝sp."

    The closest encoding I could find was "macroman" but it is still not correct. Again, not an SME, but it appears that in some cases, those non-UTF characters are supposed to be either a "ë", or not there at all. If there is uncertainty, the only thing I can suggest is to contact the data maintainers to find out the correct encoding, or whether these are indeed errors. At least then you can rule out whether it's something to do with R.