Determine if an associative array of strings or arrays is equal to another array with the same structure

I'm having these arrays, which the 1st one represents what answers has given a user on a questionnaire and the 2nd one the correct answers of each quetionnaire:

$given_answers => array(3) {
  [46] => string(2) "82" 
  [47] => string(2) "86"
  [48] => array(2) {
    [0] => string(2) "88" // key is not questionID here
    [1] => string(2) "89"  // key is not questionID here
  }
}

$correct_answers => array(3) {
  [46] => int(84)
  [47] => int(86)
  [48] => array(2) {
    [0] => int(88) // key is not questionID here
    [1] => int(91) // key is not questionID here
  }
}

NOTE: each key in both arrays represent the questionID, except from these I mention in comments. So for example questionID 46 has answerID 84 as correct answer and questionID 48 has as correct answers both 88 and 91, so the keys 0, 1 are simple array indexes in this case.

What I'm trying to do is compare both arrays and check if the answers (questionID) match for each questionID. How can I do this? I tried using array_diff() but I'm getting an error

$result = array_diff($correct_answers, $given_answers);

Severity: Notice

Message: Array to string conversion

all the answers should match exactlly the correct ones, so If I have even a single one wrong I have an error

Use the following approach:

$given_answers = [
    46=> "82",
    47=> "86",
    48=> ["88", "89"],
];

$correct_answers = [
    46=> "84",
    47=> "86",
    48=> ["88", "91"],
];

$all_matched = true;
foreach ($given_answers as $k => $ans) {
    if (!is_array($ans)) {
        if ($correct_answers[$k] != $ans) {  // comparing primitive values 
            $all_matched = false;
            break;
        }
    } else {
        // comparing arrays for equality
        if (!empty(array_diff($ans, $correct_answers[$k]))) {
            $all_matched = false;
            break;
        }
    }
}

var_dump($all_matched);  // false