Validate that a file is present

I have a JavaScript function that is looking to verify that one of two file inputs are populated, though at the moment I have an issue where validation will only work (i.e. the error message will hide) when only both input fields are populated.

Why would I be getting this?

function validateFileInput() {
    var ownImage = $('#fileField').val();
    console.log(ownImage);
    var defaultImage = $('#defImage').val();
    console.log(defaultImage);

    if (ownImage == "" || defaultImage == "") {
        $('#image_error').addClass('error-message');
        $(".error-message").css('display', 'inline-block');
        return false;
    } else {
        $('#image_error').removeClass('error-message');
        $("#image_error").css('display', 'none');
        return true;
    }
}

When logging to the console before the function is run I get

ownImage = ""
defaultImage = ""

When I then upload files to both fields I get the output below and validation passes

ownImage = "filename.jpg"
defaultImage = "4" // I'm passing a data-attribute-id here

Whereas if I only upload one image, validation fails. I want validation to pass if either of the input fields have an upload.

If you want either to be populated then if (ownImage == "" || defaultImage == "") should be if (ownImage == "" && defaultImage == "")

i.e. only display the error message if x and y are empty. If either is populated then continue.