Entity Framework 4 really slow on Nerd dinner FindByLocation modification

I have modified the nerd dinner example to find locations in the vicinity of specified position. When selecting from a flat table performance is good, but I wanted to split up the tables so I have a generic coordinates table (SDB_Geography) and also join in a table with specific data for what i call the entity type (HB_Entity).

I have made a new model called HbEntityModel which stores entity, hb and geography "sub models". Now the problem is that this query takes around 5 seconds to execute. I figured I would get a slight performance decrease by doing this but 5 seconds is just ridiculous. Any ideas on how to improve the performance with currrent table setup or do i have to go back to a monstrous flat table?

public IEnumerable<HbEntityModel> FindByLocation(float latitude, float longitude) 
{
    return (from entity in db.SDB_Entity.AsEnumerable()
                join nearest in NearestEntities(latitude, longitude, 2) 
                on entity.EntityId equals nearest.EntityId
                join hb in db.HB_Entity
                on entity.EntityId equals hb.EntityId
                join geo in db.SDB_Geography
                on entity.GeographyId equals geo.GeographyId
                select new HbEntityModel(entity, hb, geo)).AsEnumerable();
}

UPDATE

All tables contains around 14000 records.

SDB_Entity 1:0/1 SDB_Geography

SDB_Entity 1:0/1 HB_Entity

The search yields around 70 HbEntityModels.

If selecting from single table the same query takes 0.3s, using IQueryable instead of IEnumerable.

Solution

I found out how to do it with some help from Robban". See this post.

I rewrote the function to use a parameterless constructor and could then use IQueryable.

        public IQueryable<HbEntityModel> FindByLocation(float latitude, float longitude) 
    {
        return (from entity in db.SDB_Entity
                    join nearest in NearestEntities(latitude, longitude, 2) 
                    on entity.EntityId equals nearest.EntityId
                    join hb in db.HB_Entity
                    on entity.EntityId equals hb.EntityId
                    join geo in db.SDB_Geography
                    on entity.GeographyId equals geo.GeographyId
                    select new HbEntityModel() { Shared=entity, Specific=hb, Geography=geo }).AsQueryable();
    }

The query now takes around 0.4 seconds to execute which is somewhat acceptable. Hopefully things will be faster when my mean machine arrives. If someone could give me hints on how to improve the query, use a stored procedure or setup some index, i would be more than grateful.