How Do You Change The Color on The Swatches Component of JColorChooser?

I'm trying to make a drawing application involving JColorChooser Swatches Component, and I'm trying to make my UI a certain color. I was able to change the color through setting the background pretty much everywhere except for one small area around the "Recent" box. Screenshot

Any help would be appreciated, I'll paste part of my code below for context:

 //Sets up color chooser
  chooser = new JColorChooser(Color.BLACK);
  AbstractColorChooserPanel[] panels = chooser.getChooserPanels();
  for (int i = 0; i < panels.length; i++) {
     if (!panels[i].getDisplayName().equalsIgnoreCase("Swatches"))
         chooser.removeChooserPanel(panels[i]);
     else {
        panels[i].setBackground(new Color(0, 155, 228));
     }
  }
  chooser.setPreviewPanel(new JPanel());
  //Sets up size slider
  sizeSlide = new JSlider(1, 45);

  //Adds Color/Size to one control panel, adds new panel to bottom of 
  //main
  optionP = new JPanel();
  optionP.setBackground(new Color(0, 155, 228));
  optionP.setLayout(new BorderLayout());
  optionP.add(sizeSlide, BorderLayout.EAST);
  optionP.add(chooser, BorderLayout.WEST);
  this.add(optionP, BorderLayout.SOUTH);

Solution

I don't think this can be done without some sophisticated search through the JComponent tree. But even than I was not able to change the background color.

EDIT: I finally managed to change the background color for the recent panel with below code:

JColorChooser chooser = new JColorChooser(Color.BLACK);
AbstractColorChooserPanel[] panels = chooser.getChooserPanels();
for (int i = 0; i < panels.length; i++) {
    if (!panels[i].getDisplayName().equalsIgnoreCase("Swatches")) {
        chooser.removeChooserPanel(panels[i]);
    } else {
        panels[i].setBackground(new Color(0, 155, 228));
        // placing code to change recent panel background color here
        // will not work for some odd reason
        // JComponent component = (JComponent) panels[i].getComponent(0);
        // component.setBackground(new Color(0, 155, 228));
    }
}
AbstractColorChooserPanel panel = chooser.getChooserPanels()[0];
JComponent component = (JComponent) panel.getComponent(0);
component.setBackground(new Color(0, 155, 228));

Another approach worked but it will change the background of all your Panels across the application. Use the UIManager to change the Background:

UIManager.put("Panel.background", new ColorUIResource(0, 155, 228));