Remove associative elements from a flat array where the key does not exist as a value into another flat array

I have two arrays named rows and contacts.

The first array rows is like:

Array
(
    [0] => email@gmail.com
    [1] => test@gmail.com
    [2] => tester@gmail.com
    [3] => vin@gmail.com
)

The second array contacts is as:

Array
(
    [test@gmail.com] => test@gmail.com
    [ram@gmail.com] => Ram
    [vin@gmail.com] => Vinay
    [man_test@yahoo.com] => Manoj
    [homan@rediffmail.com] => Homan
)

What I want is the contacts array to be as :

Array
    (
        [ram@gmail.com] => Ram
        [man_test@yahoo.com] => Manoj
        [homan@rediffmail.com] => Homan
    )

Edit

I tried some functions like array_diff(), array_keys() etc. but they are not giving me the desired output, may be I am not able to use them correctly....!

I don't want to use loop for this purpose because the given arrays are only sample data but in real they are very huge.

Another way:

$contacts = array_diff_key($contacts, array_flip($rows));