HTML5 video (mp4 and ogv) problems in Safari and Firefox - but Chrome is all good

I have the following code:

<video width="640" height="360" controls id="video-player" poster="/movies/poster.png">
 <source src="/movies/640x360.m4v" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
 <source src="/movies/640x360.ogv" type='video/ogg; codecs="theora, vorbis"'> 
</video>

• I'm using Rails (Mongrel in development and Mongrel+Apache in production).
• Chrome (Mac and Win) can play either file (tested by one then the other source tags) whether locally or from my production servers.
• Safari (Mac and Win) can play the mp4 file fine locally but not from production.
• Firefox 3.6 won't play the video in either OS. I just get a grey cross in the middle of the video player area.
• I've made sure that both Mongrel and Apache in each case have the right MIME types set.
• From Chrome's results I know there is nothing inherently wrong with my video files or the way the files are being asked for or delivered.

For Firefox I looked at https://developer.mozilla.org/En/Using_audio_and_video_in_Firefox where it refers to an 'error' event and an 'error' attribute. It seems the 'error' event is thrown pretty well straightaway and at that time there is no error attribute. Does anyone know how to diagnose the problem?

The HTTP Content-Type for .ogg should be application/ogg (video/ogg for .ogv) and for .mp4 it should be video/mp4. You can check using the Web Sniffer.