LaTex \insertnavigation - How to change the headline colors?

I am new to beamer and I am having issues in changing the headline colors. I am preparing a presentation using beamer in overleaf. I am using the CambridgeUS template but I would like to add a headline with sections and dots representing the slides. I succeded in adding the headline using the \insertnavigation command but I am failing in changing the colors. Here is the code I am using

\documentclass{beamer}
\usepackage[utf8]{inputenc}
\usepackage{csquotes}
\usepackage{dirtytalk}
\usepackage{xcolor}
\usepackage{tgpagella}
\usepackage{adjustbox}
\usepackage{multirow}
\usepackage{dcolumn}

\usetheme{CambridgeUS}
\usecolortheme{default}
%----------------Set Colors
\xdefinecolor{scu}{RGB}{149,42,31} 
\setbeamercolor{title}{bg=scu,fg=white}

%----------------Add Headline
\setbeamertemplate{headline}{
\begin{minipage}[c][1.1cm]{\paperwidth}
        \insertnavigation{\paperwidth}
    \end{minipage}
}



%----------------Title page
\title[Title Sidebar] %optional
{Main Title}

\author[Author Sidebar] % (optional)
{
\vspace{0.2cm}
 {\large \textbf{Author}\\{\footnotesize Institution}}}


\date[August 29, 2023] % (optional)
{\footnotesize{Conference} \\
Place\\}


%---------------------------------------------------------------

\begin{document}



\begin{frame}
\titlepage
\end{frame}



\section*{Introduction}

\begin{frame}{Introducing frame}
    
\end{frame}

\section{Section 1}
\begin{frame}
    
\end{frame}

\section{Section 2}
\begin{frame}
    
\end{frame}


\section{Conclusion}
\begin{frame}{Concluding frame}
    
\end{frame}
\end{document}

I have tried some of the following commands but none worked.

\setbeamercolor{structure}{fg=white, bg=scu}
\setbeamercolor{headline}{fg=white, bg=scu}
\setbeamercolor{sidebar}{fg=white, bg=scu}

It is probably an easy mistake in the use of the colors commands but I couldn't figure out what I am missing. Any help is very appreciated.

If you look into https://github.com/josephwright/beamer/blob/main/base/themes/outer/beamerouterthememiniframes.sty#L103 you will see that beamer normally places the navigation bar in a beamercolorbox. This beamercolorbox is responsible for changing the colours.

\documentclass{beamer}
%\usepackage[utf8]{inputenc}
\usepackage{csquotes}
\usepackage{dirtytalk}
%\usepackage{xcolor}
\usepackage{tgpagella}
\usepackage{adjustbox}
\usepackage{multirow}
\usepackage{dcolumn}

\usetheme{CambridgeUS}
\usecolortheme{default}
%----------------Set Colors
\xdefinecolor{scu}{RGB}{149,42,31} 
\setbeamercolor{title}{bg=scu,fg=white}

%----------------Add Headline
\makeatletter
\setbeamertemplate{headline}{%
  \begin{beamercolorbox}[colsep=1.5pt]{upper separation line head}
  \end{beamercolorbox}
  \begin{beamercolorbox}{section in head/foot}
    \vskip2pt\insertnavigation{\paperwidth}\vskip2pt
  \end{beamercolorbox}%
%  \ifbeamer@theme@subsection%
%    \begin{beamercolorbox}[colsep=1.5pt]{middle separation line head}
%    \end{beamercolorbox}
%    \begin{beamercolorbox}[ht=2.5ex,dp=1.125ex,%
%      leftskip=.3cm,rightskip=.3cm plus1fil]{subsection in head/foot}
%      \usebeamerfont{subsection in head/foot}\insertsubsectionhead
%    \end{beamercolorbox}%
%  \fi%
  \begin{beamercolorbox}[colsep=1.5pt]{lower separation line head}
  \end{beamercolorbox}
}
\makeatother

\setbeamercolor{section in head/foot}{bg=green,fg=orange}

%----------------Title page
\title[Title Sidebar] %optional
{Main Title}

\author[Author Sidebar] % (optional)
{
\vspace{0.2cm}
 {\large \textbf{Author}\\{\footnotesize Institution}}}


\date[August 29, 2023] % (optional)
{\footnotesize{Conference} \\
Place\\}


%---------------------------------------------------------------

\begin{document}



\begin{frame}
\titlepage
\end{frame}



\section*{Introduction}

\begin{frame}{Introducing frame}
    
\end{frame}

\section{Section 1}
\begin{frame}
    
\end{frame}

\section{Section 2}
\begin{frame}
    
\end{frame}


\section{Conclusion}
\begin{frame}{Concluding frame}
    
\end{frame}
\end{document}

However instead of creating a headline yourself, I think it would be easier to simply load the miniframes outer theme:

\documentclass{beamer}
\usepackage[utf8]{inputenc}
\usepackage{csquotes}
\usepackage{dirtytalk}
\usepackage{xcolor}
\usepackage{tgpagella}
\usepackage{adjustbox}
\usepackage{multirow}
\usepackage{dcolumn}

\usetheme{CambridgeUS}
\usecolortheme{default}
%----------------Set Colors
\xdefinecolor{scu}{RGB}{149,42,31} 
\setbeamercolor{title}{bg=scu,fg=white}
\useoutertheme[subsection=false]{miniframes}

\setbeamercolor{section in head/foot}{bg=green,fg=orange}

%----------------Title page
\title[Title Sidebar] %optional
{Main Title}

\author[Author Sidebar] % (optional)
{
\vspace{0.2cm}
 {\large \textbf{Author}\\{\footnotesize Institution}}}


\date[August 29, 2023] % (optional)
{\footnotesize{Conference} \\
Place\\}


%---------------------------------------------------------------

\begin{document}



\begin{frame}
\titlepage
\end{frame}



\section*{Introduction}

\begin{frame}{Introducing frame}
    
\end{frame}

\section{Section 1}
\begin{frame}
    
\end{frame}

\section{Section 2}
\begin{frame}
    
\end{frame}


\section{Conclusion}
\begin{frame}{Concluding frame}
    
\end{frame}
\end{document}