`nops_eval()` silently drops results from open-ended string question

I administered a written exam built by the awesome exams::exams2nops() function containing multiple-choice and single-choice questions along with one open-ended string question. Scanning the PDF files from both exam sheets via nops_scan("xx.pdf") and nops_scan("yy.pdf", string = TRUE) worked like a charm. The resulting nops_scan_*.zip (containing Daten.txt) and nops_string_scan_*.zip (containing Daten2.txt) files look as they should.

However, running

ev <- nops_eval(register = "participants.csv",
                solutions = "solutions.rds",
                scans = "nops_scan_20251203064700.zip",
                string_scans = "nops_string_scan_20251203065026.zip")

only recognizes the results of the multiple-choice and single-choice results correctly. The information from the string scans is dropped without any warning or error and the scan2 column of ev consists of empty strings "".

Update: One page in the string scans yy.pdf was scanned twice!

Update

Up to version 2.4-2 nops_eval() did not handle duplicated exam IDs in the string scans properly and hence erroneously dropped the corresponding results silently.

In version 2.4-3 (development version on R-Forge at the time of writing) the problem was fixed and nops_eval() now issues the following warning:

Matching scans and string_scans is not possible with duplicated IDs.
Dropping duplicates of the following string IDs: ...

Thus, if the same page was erroneously scanned twice, dropping the duplicates is the correct thing to do and no further action is needed.

However, if the the same exam sheet was printed out several times and filled out by different examinees, then automatic matching is not possible. A workaround in this situation is to apply nops_scan() and nops_eval() repeatedly for separate batches of exams, making sure that each exam ID occurs only once in every batch.

---

The subsequent tutorial for evaluated NOPS exams with open-ended questions was written in order to find the answer above. It is not needed for the answer anymore but preserved nevertheless in case it is useful for others.

---

Exam creation

Using the exercises deriv2 (schoice), boxplots (mchoice), function (string), I set up a NOPS exam with 3 random replications.

library("exams")
set.seed(0)
exams2nops(c("deriv2.Rmd", "boxplots.Rmd", "function.Rmd"),
  dir = "nops", name = "demo", n = 3, date = "2022-02-22")

Participants

Then I create a CSV file with the info for three participants.

writeLines("registration;name;id
1234567;Jane Doe;jane_doe
0000123;Ambi Dexter;ambi_dexter
5555555;Ano Nym;ano_nym", "demo.csv")

Exam sheets

Then I printed the three exams, filled them out with the participant information above, and scanned the regular exam sheets and the string sheets.

Save these files as S0000001.png, S0000002.png, ..., T0000003.png in a subdirectory png for the subsequent code to work.

Scanning

The data from the scans can be read via:

nops_scan(Sys.glob("png/S*.png"))
nops_scan(Sys.glob("png/T*.png"), string = TRUE)

Evaluation

And then the final evaluation also works as desired:

ev <- nops_eval(
  register = "demo.csv",
  solutions = "nops/demo.rds",
  scans = Sys.glob("nops_scan_*.zip"),
  string_scans = Sys.glob("nops_string_scan_*.zip"))

The results in ev are:

       registration        name          id        exam scrambling
1234567      1234567    Jane Doe    jane_doe 22022200001         00
0000123      0000123 Ambi Dexter ambi_dexter 22022200002         00
5555555      5555555     Ano Nym     ano_nym 22022200003         00
                scan        scan2    points mark answer.1 solution.1 check.1
1234567 S0000001.png T0000001.png 1.7500000    4    00001      00100       0
0000123 S0000002.png T0000002.png 0.6666667    5    10000      01000       0
5555555 S0000003.png T0000003.png 0.2500000    5    00100      01000       0
        points.1 answer.2 solution.2   check.2  points.2 answer.3 solution.3
1234567        0    01110      11110 0.7500000 0.7500000    00000      00000
0000123        0    01001      01010 0.1666667 0.1666667    00000      00000
5555555        0    10101      00010 0.0000000 0.0000000    00000      00000
        check.3 points.3
1234567    1.00     1.00
0000123    0.50     0.50
5555555    0.25     0.25