How to subset a font so that remaining characters keep original sizes?

I am subsetting font M PLUS 1p (style medium). I need only a single character - leftwards arrow, which is unicode character u2190. During the subsetting process FontForge complained that the lookup character glyph08514 is missing (upwards arrow), so I included that one in subset as well.

I have added the resulting TTF font in my Android project (directory res/font) and referenced it from the layout file in the following way:

<androidx.gridlayout.widget.GridLayout
    android:layout_height="wrap_content"
    android:layout_width="match_parent"
    app:columnCount="3"
    app:rowCount="5"
    >
    <Button
        app:layout_row="0"
        app:layout_column="0"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="C"
        android:textSize="@dimen/font_size"
        app:layout_columnWeight="1"
        style="?android:attr/buttonBarButtonStyle"
        android:onClick="onKeyPressed"
    />
    <Button
        android:fontFamily="@font/custom"
        app:fontFamily="@font/custom"
        app:layout_row="0"
        app:layout_column="2"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="\u2190"
        android:textSize="@dimen/font_size"
        app:layout_columnWeight="1"
        style="?android:attr/buttonBarButtonStyle"
        android:onClick="onKeyPressed"
    />
    <!-- remaining buttons using the default font, just like the first one -->
</androidx.gridlayout.widget.GridLayout>

Now the arrow button is displayed 'shorter' than the other buttons (left image). This is a problem as when pressed it shows background that is not the same size that other buttons have. However, if I include the complete, unmodified, font and not it's subset, the button has a normal height (right image). This is what I am trying to achieve.

I am suspecting that some properties/dimensions get lost in the font subset. Which and how to preserve them - I have no idea. Except FontForge I have tried several on-line tools. Either I am receiving the result with the same behavior, or nothing at all (tools didn't provide download link).

Please note that setting android:layout_height="match_parent" in the first case (font subset) does not help.

It appears that FontForge is modifying the font metrics after subsetting the font to reflect the metrics of the glyphs in the subset, which have less height. If you include other glyphs in the subset that extend higher above the baseline and that extend lower below the baseline (could be one going above and below, or one going above and another going below), then the font metrics should have greater ascent and descent due to those glyphs.