Choppy JavaScript Animation of background-position-y

So the actual problem I'm having is that I want a fixed background image effect while keeping the background image wide enough to fill a container. Unfortunately, the CSS way, using background-attachment:fixed; will only position the image relative to the viewport. This creates a problem on screen sizes that have drastically different aspect ratios than the background image where it causes unwanted clipping of the background image (which is a picture of a person).

The solution I've arrived at currently is using background-size: contain to prevent clipping of the image on all screen sizes while simulating the background-attachment: fixed with JavaScript by animating background-position-y on scroll. The problem is if I then click a button that causes a scrollTop animation down the page the background-position-y setting of the image is jerky.

Any idea how to prevent this jerkiness in the animation?

// Banner scroll function
    var scrollTop = $(window).scrollTop();
    var banner = $('.hero-banner').css('background-position-y',scrollTop+'px');
    var debounced_fixBG = function(){
        var newScrollTop = $(window).scrollTop();
        banner.css('background-position-y',newScrollTop+'px');
        scrollTop = newScrollTop;
    };
    $(window).scroll(debounced_fixBG);
    
     $('.scroll-down').on('click',function(){
        $('html,body').animate({
            easing: 'swing',
            scrollTop: $('#anchor').offset().top
        },2000);
    });

.hero-banner {
  background-image: url('http://londonprogressivejournal.com/site_images/741.jpg');
  height: 383px;
  background-repeat: no-repeat;
  background-attachment: contain;
  background-position-y: top;
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<button class="scroll-down">
Scroll Down
</button>
<div class="hero-banner">
&nbsp;
</div>
<br>
<br>
<br>
<br>
<br>
<a id="anchor">Anchor</a>

Posting answer from the comments:

You can use both background-attachment:fixed; and background-size:contain; and avoid the top cut by offsetting the image with background-position-y:[height of top element];.