Svg gets a different size after appling svgPanZoom

Svg auto-resizes to fill it's column container. After it completely loads svgPanZoom is applied and I end of with a smaller image:

If I apply a height to the container it will resize leaving me with blank spaces on each side.

Problem is that when I zoom in and zoom out again the image is not centered anymore:

I think there are two approaches to solve this: 1. Recenter the image on zoomOut. 2. Use viewBox preserveAspect to make the image resize correctly without applying the size via CSS.

Both for me are a bit confusing and after several hours trying I can't make it work. This is my code:

HTML:

...
<div class="col-sm-9">
    <div class="title">
        <h1>Floor: {{ floor }}</h1>
        <div id="map-container"></div>
    </div>
</div>
...

JS:

function createNewEmbed(data){
    var embed = document.createElement('object');
    embed.setAttribute('id', 'map');
    embed.setAttribute('style', 'background-color: blue');
    embed.setAttribute('type', 'image/svg+xml');
    embed.setAttribute('data', data);
    document.getElementById('map-container').appendChild(embed)
    lastEventListener = function(){
        // Zoom and Pan
        panZoomInstance = svgPanZoom(embed, {
            zoomEnabled: true,
            controlIconsEnabled: false,
            fit: true,
            center: true,
            minZoom: 1,
            zoomScaleSensitivity: 0.4,
            beforePan: beforePan
        });

        // Zoom out
        // panZoomInstance.zoom(0.2)
        $(window).resize(function(){
            panZoomInstance.resize();
            panZoomInstance.fit();
            panZoomInstance.center();
        })

    embed.addEventListener('load', lastEventListener)
    return embed
}

function removeEmbed(){
    // Destroy svgpanzoom
    svgPanZoom(lastEmbed).destroy()
    // Remove event listener
    lastEmbed.removeEventListener('load', lastEventListener)
    // Null last event listener
    lastEventListener = null
    // Remove embed element
    document.getElementById('map-container').removeChild(lastEmbed)
    // Null reference to embed
    lastEmbed = null
}

function beforePan (oldPan, newPan){
    var stopHorizontal = false
    , stopVertical = false
    // Computed variables
    , sizes = this.getSizes()
    , gutterWidth = sizes.width
    , gutterHeight = sizes.height
    , leftLimit = -((sizes.viewBox.x + sizes.viewBox.width) * sizes.realZoom) + gutterWidth
    , rightLimit = sizes.width - gutterWidth - (sizes.viewBox.x * sizes.realZoom)
    , topLimit = -((sizes.viewBox.y + sizes.viewBox.height) * sizes.realZoom) + gutterHeight
    , bottomLimit = sizes.height - gutterHeight - (sizes.viewBox.y * sizes.realZoom)

    customPan = {}
    customPan.x = Math.max(leftLimit, Math.min(rightLimit, newPan.x))
    customPan.y = Math.max(topLimit, Math.min(bottomLimit, newPan.y))
    return customPan
}

At the start I'm doing:

window.onload = function(){
    var lastEventListener = null;

    var lastEmbedSrc = '/assets/images/6e2.svg'
        , lastEmbed = createNewEmbed(lastEmbedSrc)
        ;

Looks like using the optimized inkscape format doesn't work. regular inkscape svg and a little css did the trick:

#map {
    height: 100%;
    width: 100%;
}

I'm guessing that it has to do with the viewBox missing on this format. If anyone know other way to solve this that would be nice since the optimized version is smaller.