Handling click events inside ExpandableRecyclerView group view

I'm using this library to implement an ExpandableRecyclerView. The group view looks like this:

<android.support.v7.widget.CardView xmlns:card_view="http://schemas.android.com/apk/res-auto"
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content">

<RelativeLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent" >

    <TextView
        android:id="@+id/expandable_list_group_title"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Group Heading" />

    <ImageButton
        android:id="@+id/expandable_list_delete"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:src="@drawable/ic_delete_black_24dp"
        android:layout_alignParentEnd="true" />

</RelativeLayout>

The click events for expanding/collapsing are handled by the library and I'm adding elements to the list through a method in the Adapter. The way the library is coded, there are specific notify methods for addition and removal of items:

Please note that the traditional notifyDataSetChanged() of RecyclerView.Adapter does not work as intended. Instead Expandable RecyclerView provides a set of notify methods with the ability to inform the adapter of changes to the list of ParentListItems.

notifyParentItemInserted(int parentPosition), notifyParentItemRemoved(int parentPosition), notifyParentItemChanged(int parentPosition), notifyParentItemRangeInserted(int parentPositionStart, int itemCount)

I want to use the ImageButton to handle the deletion of the given group from the list. To do so, I must access the Adapter method notifyParentItemRemoved. Should I handle the click within the ViewHolder for the group, like so:

public class MyParentViewHolder extends ParentViewHolder {

    TextView groupTitle;
    ImageButton deleteButton;

    public MyParentViewHolder(View itemView) {
        super(itemView);
        groupTitle = (TextView) itemView.findViewById(R.id.expandable_list_group_title);
        deleteButton = (ImageButton) itemView.findViewById(R.id.expandable_list_delete);
        deleteButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                   ... // somehow delete
            }
        });
    }

public void bind(...) {
    ...
}

If so, how would I

1. Access the list items that I want to remove? The ArrayList is in the Adapter class.
2. Cause the list to update? It requires calling the Adapter notify method.

Should I have a reference to the Adapter within the ViewHolder? It does not feel right to handle it in this manner. If not, then how can I handle the click event from within the expandable group view?

Since there were no responses, I had to try to fix the issue myself. I don't know if this is the best way to go about doing this (it's probably not), but there seems to be no other choice.

From the Adapter, I pass a reference to the adapter object to the ViewHolder constructor. The click is handled within the ViewHolder itself. The adapter method is called, which further calls the notify method. This works fine.