Undo pending changes does not clean up state?

I am new to TFS and Visual Studio (I use 2017). When I create a folder, then "Undo pending changes", the new folder is gone. However, when I try to create the folder with the same name, it reports error that the name already exists. Is there a way to clean it up so that I can create the folder?

As mentioned in "Deleting a folder in TFS", you might miss the "Check in your changes" step, since it is the what deletes the folder in Source Control.

That would synchronize your local folder and what you have instructed TFS to do.