<button> vs. <input type="button"> -- which to use?

When looking at most sites (including SO), most of them use:

<input type="button" />

instead of:

<button></button>

• What are the main differences between the two, if any?
• Are there valid reasons to use one instead of the other?
• Are there valid reasons to use combine them?
• Does using <button> come with compatibility issues, seeing it is not very widely used?

Now, several years on, we have MDN in favor of <button>:

While <input> elements of type button are still perfectly valid HTML, the newer <button> element is now the favored way to create buttons. [...]

So basically, the only reason you'd use <input> for buttons now is for supporting IE<11 (see caniuse). Just take care to use type="button" if you don't want it to submit your form when you click, as this answer states.

Here's a page talking about some stylistic differences (basically you can put html into a <button></button>).

For some IE6-era information here is a page describing why people avoid <button></button> for old browsers.

Another IE problem when using <button />:

And while we're talking about IE, it's got a couple of bugs related to the width of buttons. It'll mysteriously add extra padding when you're trying to add styles, meaning you have to add a tiny hack to get things under control.