How do QA teams handle browser testing of Google Chrome?
Since Google Chrome constantly updates itself, there is a challenge in limiting test scope to a specific version. While we might limit support to IE8 and IE9, Google Chrome's version number is a moving target.
Possible approaches:
Approach #1 means everyone needs to sync up on a specific version, and edit their registry to prevent automatic updates. Furthermore, you will be releasing code for version X, while the world will have mostly auto-upgraded to version X+1.
The problem with #2 is that the browser functionality could change within dev cycles. Suppose QA completes on Monday, and code is pushed to production Wednesday. If Google releases a major automatic update on Tuesday, it could cause the code to break and no one would know until it hit production.
How do mature web development organizations handle this?
We go with option 2 for a number of reasons.
1) We try to simulate the user experience during testing (shutting off updating would simulate a deviation from the norm)
2) Would much rather find a defect in the last round of testing than have them reported by users
3) I can't think of a time where we had a serious defect introduced by testing this way. As Matt mentioned earlier it is fairly unlikely that their updates would break important functionality.
To help mitigate some of this risk we try to use selenium for our core functional automation testing to constantly test different browsers. This doesn't catch everything, but at least will find large issues (like a missing column). It also helps to be agile about these things. We can release on any required schedule (monthly, weekly, daily if needed) so if a serious issue is found we can fix it the same day.