How to handle load in a high traffic Java web application?

I am developing a high-traffic java web application. Currently I'm defining its architecture. I have some questions:

• What are the key elements that we should consider while designing the architecture to handle load?
• How to prevent web application from getting down or how to ensure high availability?

I expect hundreds of user logging in at a time. I am planning to keep data in application scope (static variables) to avoid lookups in the database. In session I am planning to store 5kB of data.

I guess key elements I would consider would be:

• Do you need cluster aware app servers?
• Do you need hardware or software based load balancers?
• What parameters will you record to determine the health of a server?
• Will you load balance your DBs?
• Will your app be I/O heavy? CPU heavy? Both?
• Are you looking to utilize a lot of web traffic? If you're averaging 5K of data per user, how many users will you be able to handle before you consider a link saturated?
• Scale up vs scale out. Determine which one your app does best and exploit it.
• Will there be a lot of DB transactions?
• Will you be using Shared Storage?

I hope this helps.