Not adding to the Build queue when a scheduled run does not complete

I have a build definition in TFS that is scheduled to run every hour 24/7. Occasionally there are times though that my build agent goes offline, or something interferes and it hangs and the build does not complete. In this case, I end up getting 2-3 or even more queued up because they did not get a chance to run but were scheduled

Was wondering how I can configure TFS to not add on to the queue if a build is still running or to kill whatever is running before kicking off a new build.

Unfortunately there isn't such a built-in feature/setting can prevent adding to the build queue. Once the build agent is online, the queued builds will run one by one...

However you can cancel and delete the queued and running builds from Build definition --> Queued tab. Filter and select all the queued and running builds and cancel them before kicking off a new build...

Please see Pause build and clear long build queue for details.