re question non-random-salt-for-password-hashes Mr Potato Head states that the use of md5 instead of SHA-512 makes generating rainbow tables easier? I'd have thought that once your rainbow table is generated that the algorithm used is irrelevant? It would make no difference to how you use the rainbow table to check for known hashs? And is there any way to know what hashing algorithm was used any way?
Edit update:
I think that proper hashing of your password tables is required, not to protect your application, but to protect everyone else where users will re-use passwords and ids.
Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes touches on this briefly. It starts out with a mini-rant about people misunderstanding rainbow tables and the actual level of danger they present, but then turns into a good discussion of the implications of password hashing algorithm performance.
On your final question, the more direct answer is that, by examining the output alone, you can only determine the number of bits in the output of the final hashing algorithm. Depending on the algorithm, this may or may not be unique, but, if the algorithm's any good, there won't be any detectable patterns that will identify it exactly. (I say "final" because the output of MD5-only will look the same as the end result of SHA512-then-MD5, since the last step is the same in both cases.)
More practically, though, anyone who can steal your password database can probably steal your source code as well, so they can just look at your source to see what your algorithm is and duplicate it for their attacks on your database.