I'm working on enabling Facebook Connect with my website, and it seems to work pretty good so far. However, loading the Facebook API library on every page of my site is bogging it way down. All the other PHP classes that I'm using compile in well under 1 second, but I'm seeing serve times ranging from 3 - 20 seconds just to get the Facebook API loaded up. Is there anything I can do to speed up Facebook Connect? Is it just because I'm on a (mt) shared server?
Facebook allows you to cache user data for 24 hours. This can help a lot when you are doing a lot of queries to Facebook for heavier data like friends. This does not help for things like profile pics which are not easy to cache but can save you a lot in some places.
Keep in mind that some of the slow down can be in the cross site javascript which really has nothing to do with your server but can really add up. An application like fiddler will help identifying where the slow down is in these cases.