I am trying to write a script to allocate moderators/second markers to essays that have already been marked but I am struggling to get it right.

There are two main conditions I have:

• As the markers and the moderators are the same group of people, no moderator should receive an essay they marked (it should be one of the other moderators);
• The assortment should be as close to equal as possible across moderators (roughly equal amounts of essays each).

Let's say markers are AA, AB, AC and AD. And we have 17 papers in total. I can randomly allocate those to the 4 people:

library(randomizr)
mod_list = complete_ra(N = 17, num_arms = 4, conditions = markers)

Say, 11 papers were allocated fine, but 5 got the same moderator as the marker:

If I then use

%>%
mutate(no.self = ifelse(moderator == "AA" & marker == "AA",
                          sample(c("AB", "AC", "AD"), 2),
                          ifelse(moderator == "AB" & marker == "AB",
                                 sample(c("AA", "AC", "AD"), 1),
                                 ifelse(moderator == "AC" & marker == "AC",
                                        sample(c("AA", "AB", "AD"), 1),
                                        ifelse(moderator == "AD" & marker == "AD",
                                               sample(c("AA", "AB", "AC"), 1),
                                               moderator)))))

the equal assortment disappears.

How can I make sure (1) every moderator needs to second mark someone else's work, yet (2) the assortment is roughly equal across moderators?

Thanks!

You can resample the repeated markers/moderators until the pairs are all different.

library(randomizr)

# brute force, resample until they are all different
shuffle <- function(x, y) {
  while(any(x == y)) {
    y <- sample(y)
  }
  y
}

i <- apply(df1[-1], 1L, \(x) x[1L] == x[2L])
df1$moderator[i] <- shuffle(df1$marker[i], df1$moderator[i])
df1[i, ]
#>    paperID marker moderator
#> 1        1     AD        AA
#> 4        4     AB        AA
#> 8        8     AA        AD
#> 17      17     AA        AB

Data

Data creation code.

library(randomizr)

set.seed(2023)
markers <- c("AA", "AB", "AC", "AD")
mark_list <- complete_ra(N = 17, num_arms = 4, conditions = markers) |> as.character()
mod_list <- complete_ra(N = 17, num_arms = 4, conditions = markers) |> as.character()
paperID <- seq.int(17)

df1 <- data.frame(paperID, marker = mark_list, moderator = mod_list)