Zod refine not displaying errors for optional array fields in react-hook-form

I'm using Zod with react-hook-form to validate a form where at least one of two fields (warehouses or clusters) must be provided. Both fields are optional arrays of strings. I'm using .refine to enforce this logic, but the error message is not showing up for the fields.

Here's my Zod schema:

const schema = z.object({
    warehouses: z.array(z.string()).optional(),
    clusters: z.array(z.string()).optional(),
})
.refine((data) => (data.warehouses?.length ?? 0) > 0 || (data.clusters?.length ?? 0) > 0, {
    message: 'At least one warehouse or cluster must be provided',
    path: ['warehouses', 'clusters'],
});

I'm using a custom FormField component with Controller from react-hook-form, but the error message is not displayed for the fields. The problem is not in my custom FormField, but in zod schema and its error handlers

Here are my FormField components:

<FormField
    form={form}
    label={'Warehouses'}
    name="warehouses"
    render={({ field, fieldState }) => (
        <AutofillMultibox
            field={field}
            choices={['South', 'East', 'West'].map((e) => ({
                label: e,
                value: e,
            }))}
        />
    )}
/>
<FormField
    form={form}
    label={'Clusters'}
    name="clusters"
    render={({ field, fieldState }) => (
        <AutofillMultibox
            field={field}
            choices={['South', 'East', 'West'].map((e) => ({
                label: e,
                value: e,
            }))}
        />
    )}
/>

Well, refine only works after all values have passed the default type checks. I expected an error earlier, just like the "Field is required" validation for other inputs. So I had to stop using refine and replace it with:

locations: z.discriminatedUnion('type', [
        z.object({
            type: z.literal('warehouses'),
            warehouses: z.array(z.string()).min(1, 'Choose at least 1 warehouse'),
            clusters: z.array(z.string()).default([]),
        }),
        z.object({
            type: z.literal('clusters'),
            clusters: z.array(z.string()).min(1, 'Choose at least 1 cluster'),
            warehouses: z.array(z.string()).default([]),
        }),
    ]),