Getting a panel within a panel with mouse position

I've been looking for this thing... It should be working yet it is not. There must be something I don't get understand or that I'm missing. It's quite a simple problem but I can't seem to solve it.

I got Panel1 and Panel2 as shown in this picture.

I want to catch when mouse is over Panel2 within Panel1 MouseLeave event. My code goes like this :

Private Sub Panel1_MouseLeave(sender As Object, e As EventArgs) Handles Panel1.MouseLeave

    If sender.ClientRectangle.Contains(PointToClient(Control.MousePosition)) Then
        For Each ctrl As Object In sender.controls
            If ctrl.ClientRectangle.Contains(PointToClient(Control.MousePosition)) Then Exit Sub
        Next
        If Not IsNothing(sender.BackgroundImage) Then sender.BackgroundImage = Nothing
    End If

End Sub

Private Sub Panel2_MouseLeave(sender As Object, e As EventArgs) Handles Panel2.MouseLeave

    If Not sender.ClientRectangle.Contains(PointToClient(Control.MousePosition)) Then
        If Not IsNothing(sender.BackgroundImage) Then sender.BackgroundImage = Nothing
    End If

End Sub

I'm successfully getting into the first if, but the 2nd one within the For Each just never equals true. So I thought maybe there was a problem with the 2nd panel, so I tried placing the same code for Panel2 MouseLeave, but it's working just fine.

I really need this code to work for a big control flickering problem I'm having.

Thanks to Hans Passant for the hint. I just had to call the PointToClient with the right control :

Private Sub Panel1_MouseLeave(sender As Object, e As EventArgs) Handles Panel1.MouseLeave

    If sender.ClientRectangle.Contains(Panel1.PointToClient(Control.MousePosition)) Then
        For Each ctrl As Object In sender.controls
            If ctrl.ClientRectangle.Contains(ctrl.PointToClient(Control.MousePosition)) Then Exit Sub
        Next
        If Not IsNothing(sender.BackgroundImage) Then sender.BackgroundImage = Nothing
    End If

End Sub