How do I, for example, change the color of the player?
Yes! The HTML5 audio tag with the "controls" attribute uses the browser's default player. You can customize it to your liking by not using the browser controls, but rolling your own controls and talking to the audio API via javascript.
Luckily, other people have already done this. My favorite player right now is jPlayer, it is very stylable and works great. Check it out.