Problem deleting all sub-menu items correctly in CMFCMenuButton

I am getting myself confused.

I have a menu:

IDR_MENU_POPUP_MANAGE_GROUPS MENU
BEGIN
    POPUP "__MANAGE_GROUP__"
    BEGIN
        MENUITEM "Add Group",                   ID_POPUP_ADD_GROUP
        POPUP "Edit Group"
        BEGIN
            MENUITEM "__EDIT__",                    ID_POPUP_EDIT_GROUP_BASE
        END
        POPUP "Delete Group"
        BEGIN
            MENUITEM "__DELETE__",                  ID_POPUP_DELETE_GROUP_BASE
        END
    END
END

This is used by a CMFCMenuButton. At runtime I dynamically delete the two submenus like this:

CMenu* pMenu = m_menuManageGroups.GetSubMenu(0);
CMenu* pSubMenu = nullptr;

pSubMenu = pMenu->GetSubMenu(1);
pMenu->RemoveMenu(ID_POPUP_EDIT_GROUP_BASE, MF_BYCOMMAND);

In the first instance that works and the Edit sub-menu is not there in my menu button.

But then I add some menu items to the sub menu:

for (auto& groupInfo : mapGroups)
{
    // first:  Group Name
    // second: Group Id
    const auto iMenuItemId = iMenuBase + groupInfo.second;

    pSubMenu->AppendMenu(MF_STRING, iMenuItemId, groupInfo.first);

}

They get added and I see them in the sub menu of the button.

Now, if for some reason I run this code a second time it ends up adding the new menu items to the bottom of the existing sub menu in the menu button. Why? I thought RemoveMenu would delete all existing sub menu items.

I managed to do it like this:

const auto menuCount = pSubMenu->GetMenuItemCount();
for (int iMenuItem = 0; iMenuItem < menuCount; iMenuItem++)
{
    pSubMenu->DeleteMenu(0, MF_BYPOSITION);
}

But I still don't understand why I had to do this because I thought:

pMenu->RemoveMenu(iMenuBase, MF_BYCOMMAND);

... would get rid of all the sub-menu items in the flyout.