[SOLVED] Custom notification for android with buttons

So I'm trying to build a notification that has some text and two buttons on the right side (not at the bottom like a gmail/whatsapp notification). I created the layout for the same, but when I set a remote view on the notification, it only displays the title text view and the two buttons. It doesn't display the image view or the text view below the title. Also, the margins I've specified don't get applied either. I'm pasting the xml as well as the code below:

custom_notification.xml

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content">

<ImageView
    android:id="@+id/img_logo"
    android:layout_width="36dp"
    android:layout_height="36dp"
    android:layout_margin="10dp"
    android:scaleType="centerCrop"
    android:src="@drawable/ic_launcher"/>

<TextView
    android:id="@+id/txt_title"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_toEndOf="@+id/img_logo"
    android:layout_alignTop="@+id/img_logo"
    android:text="Would you like to connect with this person?"
    style="@style/NotificationTitle"
    android:textColor="@android:color/darker_gray" />

<TextView
    android:id="@+id/txt_contact_name"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_alignLeft="@id/txt_title"
    android:layout_below="@id/txt_title"
    android:text="Lord Voldemort"
    style="@style/NotificationText"
    android:textColor="@android:color/black"
    android:textStyle="bold"/>

<ImageButton
    android:id="@+id/img_btn_reject_suggestion"
    android:layout_width="36dp"
    android:layout_height="36dp"
    android:scaleType="centerCrop"
    android:layout_alignTop="@+id/img_logo"
    android:src="@drawable/ic_reject_suggestion"
    android:background="@android:color/transparent"
    android:layout_toStartOf="@+id/img_btn_accept_suggestion"
    android:layout_marginRight="10dp"
    android:layout_centerVertical="true"/>

<ImageButton
    android:id="@+id/img_btn_accept_suggestion"
    android:layout_width="36dp"
    android:layout_height="36dp"
    android:layout_alignTop="@+id/img_logo"
    android:layout_marginRight="10dp"
    android:scaleType="centerCrop"
    android:src="@drawable/ic_accept_suggestion"
    android:background="@android:color/transparent"
    android:layout_alignParentRight="true"
    android:layout_centerVertical="true"/>

    </RelativeLayout>

and here is the method that creates the notification:

NotificationManager mNotificationManager = (NotificationManager) context.getSystemService(context.NOTIFICATION_SERVICE);

    Notification.Builder mNotificationBuilder = new Notification.Builder(context);
    RemoteViews contentView = new RemoteViews(context.getPackageName(), R.layout.custom_notification);

    mNotificationBuilder.setSmallIcon(R.drawable.ic_small_notification)
            .setDefaults(Notification.DEFAULT_ALL)
            .setContent(contentView)
            .setAutoCancel(true);


    Notification notification = mNotificationBuilder.build();

    mNotificationManager.notify(MANUAL_NOTIFICATION_ID, notification);

I did a lot of digging around, but cannot figure out why this doesn't work. Do let me know if any more info is needed. :)