<button> vs. <input type="button"> -- which to use?

When looking at most sites (including SO), most of them use:

<input type="button" />

instead of:

<button></button>

• What are the main differences between the two, if any?
• Are there valid reasons to use one instead of the other?
• Are there valid reasons to use combine them?
• Does using <button> come with compatibility issues, seeing it is not very widely used?

• Here's a page describing the differences (basically you can put html into a <button></button>)
• And another page describing why people avoid <button></button> (Hint: IE6)

Another IE problem when using <button />:

And while we're talking about IE, it's got a couple of bugs related to the width of buttons. It'll mysteriously add extra padding when you're trying to add styles, meaning you have to add a tiny hack to get things under control.