Switching between earpiece and speakerphone on button press

I am trying to play audio from both the speakerphone and earpiece by having a button toggle between the two. The problem is that I am trying to default the audio to play from the earpiece, but nothing comes out. Then when I press the button to toggle to speakerphone, still no audio plays. I am playing from a local raw file.

I have android.permission.MODIFY_AUDIO_SETTINGS in the Manifest as well.

Here is my code:

@Override
public void onCreate(Bundle savedInstanceState)
{
    super.onCreate(savedInstanceState);

    context = getActivity().getBaseContext();

    am = (AudioManager) context.getSystemService(Context.AUDIO_SERVICE);

    am.setMode(AudioManager.MODE_IN_CALL);
    am.setSpeakerphoneOn(false);
    am.setBluetoothScoOn(true);
    speakerON = false;
}

@Override
public void onClick(View v)
{               
    switch (v.getId())
    {

        case R.id.buttonSpeaker:
            if(!speakerON)//speaker off
            {
                speakerON = true;
                am.setMode(AudioManager.MODE_NORMAL);
                am.setSpeakerphoneOn(true);
                am.setBluetoothScoOn(false); 
                speaker.setCompoundDrawablesWithIntrinsicBounds(android.R.drawable.ic_lock_silent_mode_off, 0, 0, 0);
            }
            else
            {
                speakerON = false;
                am.setMode(AudioManager.MODE_IN_CALL);
                am.setSpeakerphoneOn(false);
                am.setBluetoothScoOn(true);
                speaker.setCompoundDrawablesWithIntrinsicBounds(android.R.drawable.ic_lock_silent_mode, 0, 0, 0);
            }

            break;

    }
}

Here is how I am setting up the MediaPlayer:

mediaPlayer = MediaPlayer.create(getActivity().getBaseContext(), R.raw.test_message);
mediaPlayer.setAudioStreamType(AudioManager.STREAM_VOICE_CALL);
mediaPlayer.start();

It turns out that I had set the mode wrong.

Here is the updated media player:

mediaPlayer = MediaPlayer.create(getActivity().getBaseContext(), R.raw.test_message);
mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
mediaPlayer.start();

And then I set the mode for the audio manager to :

context = getActivity().getBaseContext();

am = (AudioManager) context.getSystemService(Context.AUDIO_SERVICE);        
am.setMode(AudioManager.MODE_IN_CALL);
am.setSpeakerphoneOn(false);

And then it worked. So make sure that the media player and audio manager are in the same mode.