Finding what checkin broke a unit test

Some checkin over the last three days broke one of our unit tests (and why is not at all clear).

I'm assuming the best way to find which check in is to pull the code in the branch as of three days ago, verify the test passes, then keep pulling for a checkin halfway between works & fails to find the checkin that caused the problem.

Which leads to the questions:

1. How do I get a list of all IDs (?) of code checked in to the branch over the last three days.
2. How do I then pull that copy of the code (for that specific ID)?

We use Git via VSO (now Azure DevOps) if that matters.

You are describing exactly what git bisect does. Then all you need is to identify one working commit (before the bug was introduced) and one broken commit (presumably the current one).

Now, the whole point of bisect is to find when things were last good (or first bad) because you don't know. So when you identify "one working commit", having it be more recent is slightly quicker, but not enough as to matter. (Especially because any time you spend trying to be more precise in setting the starting point, is time that bisect could be spending more efficiently finding exactly the commit where the bug was introduced.)

So if you know that a week ago, you locally had a good (pre-bug) copy checked out on master, you can give

master@{1 week ago}

This relies on the local reflog, so only works from a repository that had a good copy checked out at that time. If that's not practical, you can use

git rev-list --until='1 week ago' -n1 master

to get the last commit created (according to the commit metadata) at least one week ago. So something like

git bisect start $(git rev-list --until='1 week ago' -n1 master) master

For either of the above notations, you can use 3 days instead of 1 week and it would be slightly more efficient. I.e., you might expect one or two fewer searches before the bug is pinpointed.

So there's not a lot of value in trying to pre-test whether a commit has the bug if you're not sure; you can just use an older commit instead. But if for whatever reason you really want to, you can use the same notations to check out a commit from a given time, and then test it however you like.

A bit of an aside, but do beware - it is quite popular these days to make "cleaner" histories by rebasing every branch before integrating it into the mainline. And while these "cleaner" histories are indeed more linear and simple, they are also in fact more likely to be dirty in the sense that they contain commits that have never been tested. If one of those commits happens to fail unit tests for reasons other than your bug—such as a merge inconsistency that got washed out in a later commit, then using bisect is suddenly not as easy, since merely the green/red status of your unit test suite will not tell you whether your bug is present.

But that would be a problem for the bug-searching strategy you describe in general (not just for bisect). So if tracking the bug through history is viable at all, then bisect is the best way to do it.