If I create a blank Mac XCode project and layout 500 simple NSView objects side by side in the main window it loads pretty damn fast. If I set wantsLayer=YES on each subview, performance dramatically drops, by several seconds. Why is this the case conceptually? It seems that layers would be faster not slower than regular old NSViews.
You're giving the system more work to do by layer-backing so many views. Layer-backing allows graphic acceleration (for drawing) but it adds a bit of overhead to things like layout, not to mention just creating them and putting them on screen. If used properly, it's not really much of a problem.
Typically, if you had so many "things" to manage on screen at once, you'd have one layer-backed hosting view that manages its own tree of sublayers. "But what about view-based table views?" you ask. Trickery, trickery, I say! Table views don't actually keep all the cell views they manage around; they efficiently reuse them, keeping around only enough to represent what's on screen and/or animating around.
So I'd say this isn't really a problem since it's not a particularly good approach to throw 500+ layer-backed views up for layout and drawing to begin with. :-)