How to delete a single record from pivot table

I have made my own Access Control List with Laravel 9 and basically, I made also a Many To Many relationships between User & Role:

User Model:

public function roles()
    {
        return $this->belongsToMany(Role::class);
    }

Role Model:

public function users()
    {
        return $this->belongsToMany(User::class);
    }

Then I created a page to see the current roles of user admins:

$roles = Role::latest()->with('users')->paginate(20);
return view('Admin.levelAdmins.all' , compact('roles'));

And in the Blade, I put this as <tbody>:

@foreach($roles as $role)
    @if(count($role->users))
        @foreach($role->users as $user)
            <tr>
                <td>{{ $user->name }}</td>
                <td>{{ $user->email }}</td>
                <td>{{ $role->name }} - {{ $role->label }}</td>
                <td>
                    <form action="{{ route('adm.level.destroy'  , ['user' => $user->id]) }}" method="post">
                        {{ method_field('delete') }}
                        {{ csrf_field() }}
                        <div class="btn-group btn-group-xs">
                            <a href="{{ route('adm.level.edit' , ['user' => $user->id]) }}"  class="btn btn-primary">Edit</a>
                            <button type="submit" class="btn btn-danger">Delete</button>
                        </div>
                    </form>
                </td>
            </tr>
        @endforeach
    @endif
@endforeach

And it properly shows the results like this:

But my problem is with the DELETE button which should delete the related role. But because the pivot table does not have primary table id, I don't know how to do that!

Firsly I put this as the action:

public function destroy(User $user)
    {
        $user->roles()->detach();
        return redirect()->route('adm.level.index');
    }

But this is wrong since it deletes all the roles of the user and not just one of them.

So how can I delete a role of a user from this table with Eloquent relationships?

Here is also my role_user table structures:

Schema::create('role_user', function (Blueprint $table) {
            $table->unsignedBigInteger('role_id');
            $table->unsignedBigInteger('user_id');
            $table->timestamps();

            $table->foreign('role_id')->references('id')->on('roles')->onDelete('cascade');
            $table->foreign('user_id')->references('id')->on('users')->onDelete('cascade');

            $table->primary(['role_id','user_id']);
        });

I would really appreciate any idea or suggestion coming from you guys...

Thanks in advance.

You also need to pass the role_id from the form

Then you can do something like this

$user->roles()->detach($role_id);