Laravel relationships attach if not already attached

Is there a quick way of attaching relationships if they're not already attached. I am using this code to update relations of a model;

        if (!empty($request->get('roles')) && is_array($request->get('roles'))) {
            $message->Roles()->attach($request->get('roles'));
        }
        if (!empty($request->get('users')) && is_array($request->get('users'))) {
            $message->Users()->attach($request->get('users'));
        }

But I am getting this error which I am this error;

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-41' for key 'PRIMARY' (SQL: insert into message_user (message_id, user_id) values (1, 41), (1, 42), (1, 43), (1, 44), (1, 45), (1, 46), (1, 47), (1, 48), (1, 49), (1, 50))

I want to avoid going through a very long list of array check which users are not already attached and attaching them. Also let me know if this is the only way.

I am thinking something like;

$message->Users()->attachIfNotAttached($request->get('users'));

Something like this might work:

Get the IDs that are already attached:

$attachedIds = $message->Users()->whereIn('id', $request->get('users'))->pluck('id');

Remove the attached IDs from the request array:

$newIds = array_diff($request->get('users'), $attachedIds);

Attach the new IDs:

$message->Users()->attach($newIds);