setBackgroundResource in onItemCheckStateChanged doesn't work for the first selection

I am working on an android project where I have a grid view and I am setting it up with a MultiChoiceModeListener.

When I long click, it successfully creates the contextual action bar but setting the setBackgroundResource on the initial item that creates the CAB doesn't have any affect, however all subsequent items that are selected, the background resource is successfully applied. Below is the code I am using:

gridView.setChoiceMode(AbsListView.CHOICE_MODE_MULTIPLE_MODAL);
        gridView.setMultiChoiceModeListener(new AbsListView.MultiChoiceModeListener()
        {

            @Override
            public void onItemCheckedStateChanged(ActionMode mode, int position, long id, boolean checked)
            {
                Log.d("DirectoryPicker-CAB", "CheckedStateChanged");
                //findViewById((int)gridView.getItemIdAtPosition(position)).setBackgroundColor(ContextCompat.getColor(DirectoryPicker.this, R.color.appPrimaryColour));
                //Toast.makeText(DirectoryPicker.this, "Item CAB Clicked: Checked: " + (checked ? "true" : "false"), Toast.LENGTH_LONG).show();
                Toast.makeText(DirectoryPicker.this, "Position: " + position, Toast.LENGTH_LONG).show();
                if (checked)
                {

                    //gridView.getChildAt(position).setBackgroundColor(ContextCompat.getColor(DirectoryPicker.this, R.color.appPrimaryColour));
                    Log.d("DirectoryPicker-View", gridView.getChildAt(position).toString());
                    gridView.getChildAt(position).setBackgroundResource(R.color.appPrimaryColour);
                    mode.invalidate();


                    directoryAdapter.getItem(position).setSelected(true);
                    selectedDirectories.add(directoryAdapter.getItem(position));

                }
                else
                {
                    directoryAdapter.getItem(position).setSelected(false);
                    FileManager.DirectoryOrFileInfo selectedDirectory = directoryAdapter.getItem(position);
                    for (int i = 0; i < selectedDirectories.size(); i++)
                    {
                        if (selectedDirectories.get(i) == selectedDirectory)
                        {
                            selectedDirectories.remove(i);
                            selectedDirectory.setSelected(false);
                        }
                    }
                    gridView.getChildAt(position).setBackgroundColor(ContextCompat.getColor(DirectoryPicker.this, android.R.color.transparent));
                }
                mode.setTitle(selectedDirectories.size() + " selected");
            }

            @Override
            public boolean onCreateActionMode(ActionMode mode, Menu menu)
            {
                MenuInflater inflater = getMenuInflater();
                inflater.inflate(R.menu.file_dir_cab_menu, menu);
                Log.d("DirectoryPicker-CAB", "OnActionModeCreate");
                return true;
            }

            @Override
            public boolean onPrepareActionMode(ActionMode mode, Menu menu)
            {
                return true;
            }

            @Override
            public boolean onActionItemClicked(ActionMode mode, MenuItem item)
            {
                return false;
            }

            @Override
            public void onDestroyActionMode(ActionMode mode)
            {
                selectedDirectories.clear();
            }
        });

Why does the background resource not get successfully applied for the first item that is long clicked that creates the CAB, but it works fine for all subsequent selections.

I've figured why this isn't working now. When the CAB is created, it triggers the getView function to be called again in my adapter so it ends up resetting the view back to how it was again.

I've therefore changed my adapter instead in the getView method so it checks whether the item is marked as checked, and it sets the background resource.