rr-exams

Having the same exercise in an exam multiple times and ensuring different generations


I have a lot of very similarly worded questions and I didn't wanna make an exercise for each. In an exam I wanna give each student, let's say, k of those questions, so I'll have to call the exercise thrice per exam. Since I don't want students to get the same questions multiple times, I figured I'd select a new sample of k questions every k calls. I've put a counter into place that counts to k, but run into some trouble with exams2nops running the code twice and the counter therefore starting at 2. This means the last question could be a repeat from the first k-1. I've come up with a workaround of sorts, but it ain't very elegant. Would appreciate a more elegant method.

This is the exercise I've now come up with (replaced the exact questions with some dummy text as to deflate the length of this).

```{r data generation, echo = FALSE, results = "hide"}
## parameters

k <- 3

envir <- .GlobalEnv
N <- envir$N

if (length(N)==0) {
list1 <- envir$list1 <- c("Option1","Option2","Option3","Option4","Option5","Option6","Option7","Option8","Option9","Option10","Option11","Option12","Option13","Option14","Option15")
list2 <- envir$list2 <- rep(c("001","010","100"),each=5)}

if (length(N)!=k)  {
  N <- envir$N <- sample(1:15,size=k)

  envir$no <- 0
} # Starts the counter over again and generates a new sample whenever a rotation is through.

envir$no <- envir$no+1

if (envir$no==k) N <- envir$N <- 1:100
# The 1:100 is kinda random. Just needs to be larger than any reasonable k.

if (envir$no==2&length(envir$M)==0) 
{
  envir$no <- 1
  envir$M <- 0
}# This is my solution to the counter issue. When the counter first hits 2, it's set back to 1, but because M exists afterwards, this is never called again.

```

Question
========

`r list1[N[no]]`

Answerlist
----------
* A
* B
* C

Meta-information
================
extype: schoice
exsolution: `r list2[N[no]]`
exname: questions
exshuffle: 3

Solution

  • Using a common environment to share information:

    I'm not sure whether I follow all of the details of your code but I think that you can do what you want to do by setting the envir = ... argument of the various exams2xyz() interfaces. Doing so you re-use the same environment for running the R code when knit-ting the exercises. Thus, you can store the information which of your options was used in previous runs and exclude them from subsequent runs.

    Determining when a new iteration starts:

    Then you just have to reset the variable with this information after a certain number of runs. If you don't want to fix the number of times you will run the exercise within the exercise, you can use the match_exams_iteration() convenience function. This tells you in which of the n runs of exams2xyz() you currently are. (Remark: The function was introduced in version 2.4-1 of the package but erroneously not exported in the namespace, yet. So we need the triple colon ::: to access it. We'll export it in version 2.4-2.)

    Demo exercise:

    Because I wasn't sure about all details of the exercise from your question, I'm introducing a new exercise superhero.Rmd here. It contains seven different multiple-choice tasks about different groups of superheroes.

    ```{r, include=FALSE}
    ## seven superhero multiple-choice tasks
    tasks <- rbind(
      c("Which of the following characters is part of the Justice League?", "Batman", "Aquaman", "Wonder Woman", "Wolverine", "Black Widow", "11100"),
      c("Which of the following characters is part of the Avengers?", "Iron Man", "Ant-Man", "Superman", "Magneto", "Deadpool", "11000"),
      c("Which of the following characters is part of the X-Men?", "Cyclops", "Beast", "Spider-Man", "Captain America", "The Flash", "11000"),
      c("Which of the following characters is part of the Guardians of the Galaxy?", "Star-Lord", "Groot", "Nebula", "Professor X", "Cyborg", "11100"),
      c("Which of the following villains are adversaries of Batman?", "Penguin", "Bane", "Poison Ivy", "Riddler", "Thanos", "11110"),
      c("Which of the following villains are adversaries of the Avengers?", "Ultron", "Lex Luthor", "Joker", "Harley Quinn", "General Zod", "10000"),
      c("Which of the following characters are romantic interests of Spider-Man?", "Mary Jane Watson", "Gwen Stacy", "Selina Kyle", "Lois Lane", "Pepper Potts", "11000")
    )
    colnames(tasks) <- c("question", paste0("answer", 1:5), "solution")
    
    ## initialization of iteration and reset the task id used
    if(!exists("iter")) iter <- 0
    if(exams:::match_exams_iteration() > iter) id_used <- NULL
    
    ## query current iteration and sample task id
    iter <- exams:::match_exams_iteration()
    id <- sample(setdiff(1:nrow(tasks), id_used), 1)
    
    ## update the task id used so far
    id_used <- c(id_used, id)
    ```
    
    Question
    ========
    Iteration: `r iter`.
    
    Task IDs used: `r paste(id_used, collapse = ", ")`.
    
    `r tasks[id, "question"]`
    
    ```{r, echo = FALSE, results = "asis"}
    answerlist(tasks[id, 2:6], markup = "markdown")
    ```
    
    Meta-information
    ================
    exname: superhero
    extype: mchoice
    exsolution: `r tasks[id, "solution"]`
    exshuffle: TRUE
    

    Note that in order to facilitate understanding what is going on, I display both the current iteration and the task IDs used so far in the question.

    Usage with exams2nops:

    To show that this exercise can be used repeatedly within the same exam, I set up an exam where the exercise is included five times. And then I generate three random versions of a NOPS exam with these five exercises.

    library("exams")
    set.seed(0)
    shared_env <- new.env()
    exams2nops(rep("superhero.Rmd", 5), n = 3, env = shared_env)
    

    Screenshots of the resulting PDFs are linked below (click to view the large version). This shows that the first screenshot has always iteration 1 and how the selected task IDs evolve. Analogously for the second and third screenshot.

    nops1.pdf screenshot nops2.pdf screenshot nops3.pdf screenshot

    When you don't share the environment, then the exercise also works but it always samples one of the seven tasks randomly. Note also that the .GlobalEnv is not affected by any of this. If you want that then you could also use envir = .GlobalEnv.