AVIERR_MEMORY error occurs during AVIStreamSetFormat() call

I have a Delphi 6 application that makes AVI movies. I have gotten to the point that I can write AVI files that have uncompressed video frames. When I try to create a compressed video stream I get an AVIERR_MEMORY error when I call AVIStreamSetFormat():

hr := AVIStreamSetFormat(
        FAvi_.thePsCompressed, 
        0, 
        @dsBmih, 
        dsBmih.biSize + dsBmih.biClrUsed * sizeof(RGBQUAD));

Why am I getting this error and what can I do to fix it?

Here is the contents of the compression options data structure after I select the Cinepak Code by Radius compressor, using a call to AVISaveOptions(). Note, if I select a different compressor I still get the error:

fccType: 0
fccHandler: 1684633187
dwKeyFrameEvery: 0
dwQuality: 8800
dwBytesPerSecond: 0
dwFlags: 8
lpFormat: nil
cbFormat: 0
lpParms: nil
cbParms: 4
dwInterleaveEvery: 0

Here is the contents of the TBitmapHeaderInfo data structure:

biSize: 40
biWidth: 320
biHeight: -240
biPlanes: 1
biBitcount: 32
biCompression: 0
biSizeImage: 307200
biXPelsPerMeter: 0
biYPelsPerMeter: 0
biClrUsed: 0
biClrImportant: 0

Here is the method that contains the call to AVIStreamSetFormat():

function TAviWriterWithCompression.compressionDirect(
                                opts: PAVICOMPRESSOPTIONS;
                                bShowDialog: boolean;
                                hparent: HWND;
                                dsBmih: TBitmapInfoHeader;
                                sizeImage: integer;
                                DIBValues: Pointer;
                                framesPerSecond: integer): HRESULT;
var
    lastErr: string;
    hr: HRESULT;
    myopts: TAVICOMPRESSOPTIONS;
    aopts: array[0..0] of PAVICOMPRESSOPTIONS;
    res: Bool;
begin
    if not FIsVirginFile then
    begin
        // The output file already has at least one audio or video frame so
        //  setting or changing the compression format is not allowed.
        Result := LongInt(AVIERR_ERROR);

        // =========================== EXIT POINT ==============
        exit;
    end; // if not Assigned(FAvi_) then

    if not Assigned(FAvi_) then
    begin
        Result := LongInt(AVIERR_BADHANDLE);

        // =========================== EXIT POINT ==============
        exit;
    end; // if not Assigned(FAvi_) then

    // Check the utility object for an error.
    if (FAvi_.iserr) then
    begin
        Result := LongInt(AVIERR_ERROR);
        // =========================== EXIT POINT ==============
        exit;
    end; // if (FAvi_.iserr) then

    // Make sure the compressor has not already been selected.
    if Assigned(FAvi_.thePsCompressed) then
    begin
        Result := LongInt(AVIERR_COMPRESSOR);
        // =========================== EXIT POINT ==============
        exit;
    end; // if (FAvi_.iserr) then

    // create the stream, if it wasn't there before
    if not Assigned(FAvi_.thePs) then
    begin
        hr := createVideoStream(dsBmih, framesPerSecond);

        if hr <> AVIERR_OK then
        begin
            Result := hr;

            // Set the error flag in our utility object.
            FAvi_.iserr := true;

            // =========================== EXIT POINT ==============
            exit;
        end; // if hr <> AVIERR_OK then
    end; // if not Assigned(FAvi_.thePs) then

    // set the compression, prompting dialog if necessary
    if not Assigned(FAvi_.thePsCompressed) then
    begin
        ZeroMemory(@myopts, sizeof(myopts));

        if Assigned(opts) then
            // Use the provided compressor options
            aopts[0] := opts
        else
            // Use our initialized (empty) variable.
            aopts[0] := @myopts;

        // Does the caller want to show the compressor dialog box?
        if (bShowDialog) then
        begin
            res := AVISaveOptions(hparent, 0, 1, FAvi_.thePs, aopts[0]);

            if not res then
            begin
                AVISaveOptionsFree(1, aopts[0]);

                // Set the error flag.
                FAvi_.iserr := true;

                Result := LongInt(AVIERR_USERABORT);

                // =========================== EXIT POINT ==============
                exit;
            end; // if res = 0 then
        end; // if (bShowDialog) then

        hr := AVIMakeCompressedStream(FAvi_.thePsCompressed, FAvi_.thePs, aopts[0], nil);

        if hr <> AVIERR_OK then
        begin
            Result := hr;

            // Set the error flag in our utility object.
            FAvi_.iserr := true;

            // =========================== EXIT POINT ==============
            exit;
        end; // if hr <> AVIERR_OK then

        AVISaveOptionsFree(1, aopts[0]);
        postDiagMsg('Avi::compression after AVISaveOptionsFree()');

        // >>>> This is where I get the AVIERR_MEMORY error.
        hr := AVIStreamSetFormat(FAvi_.thePsCompressed, 0, @dsBmih, dsBmih.biSize + dsBmih.biClrUsed * sizeof(RGBQUAD));

        if hr <> AVIERR_OK then
        begin
            Result := hr;

            // Set the error flag in our utility object.
            FAvi_.iserr := true;

            // =========================== EXIT POINT ==============
            exit;
        end; // if hr <> AVIERR_OK then
    end; // if not Assigned(FAvi_.thePsCompressed) then

    Result := AVIERR_OK;
end;

It might be caused by the negative value of the biHeight from the TBitmapHeaderInfo structure, what means the bit values stored in DIB section represents the vertically flipped image.

Some codecs might not be able to resolve this. So try to make sure the size and bit depth of the image header can be processed by your selected video stream compressor. You have to use only supported image formats and if your codec doesn't support flipped images, then you'll have to flip the DIB image data and modify the image headers manually by your own.