How to toggle active class on each of Bootstrap btn-grruop

I have a bunch of buttons inside a btn-group div which I have to set them all in active format ans ONLY toggle the class when the user clicks on the button.

Here is a demo, in which you can see after removing active class from the buttons they still keeps a light grey background. What is this? How can I get rid of that?

<div class="btn-group">
    <button type="button" class="btn btn-default active">1</button>
    <button type="button" class="btn btn-default active">2</button>
    <button type="button" class="btn btn-default active">3</button>
    <button type="button" class="btn btn-default active">4</button>
    <button type="button" class="btn btn-default active">5</button>
    <button type="button" class="btn btn-default active">6</button>
</div>

$(document).ready(function () {
    $("div").children().click(function () {
        $(this).toggleClass("active");
    });
});

How can I get rid of the darker grey at the top of the active class?

If you want to remove the grey color in active buttons, just override it as follows:

.active{
  background-color:white !important;
}

Note:

How can I get rid of the darker grey at the top of the active class?

Well, if you are referring to the highlighted button in the second image, it doesn't have active class, it's a non-active class. by setting all buttons as active initially, on first click it simply becomes inactive. You might want to think about what you are doing.

Update

If you want to change the grey color you can do so by adding another class and toggling it as follows:

.select{
   background-color:white !important;
}
$("div").children().click(function () {
    $(this).toggleClass("select");
});

Working fiddle.