box-sizing: border-box with no declared height/width

I'm trying to understand how box-sizing: border-box work in the code below. When the height or width is set (no padding), it works as intended (border appears inside the div). But if you only use the padding to create the dimension of the div, it does not work. Can someone explain why? Here's the demo:

div.test {
  background-color: red;
  box-sizing: border-box;
  display: inline-block;
  border: 5px solid;
  text-align: center;
  padding: 50px;
  vertical-align: middle;
  // height: 100px;
  // width: 100px;
}

div.test:first-of-type {
  border: none;
}

<div class="test">aa</div>
<div class="test">aa</div>

https://codepen.io/anon/pen/bxaBER

TL;DR

An idea is to keep border for both. Instead of none simply make the color transparent so that the size (including border + padding) will always be the same for both.

div.test {
  background-color: red;
  box-sizing: border-box;
  display: inline-block;
  border: 5px solid;
  text-align: center;
  padding: 50px;
}

div.test:first-of-type {
  border-color: transparent;
}

<div class="test">aa</div>
<div class="test">aa</div>

Why?

When setting height/width you explictly define that both sould have fixed size and this size will include border, padding and content. As we can read in the documentation:

border-box

tells the browser to account for any border and padding in the values you specify for an element's width and height. If you set an element's width to 100 pixels, that 100 pixels will include any border or padding you added, and the content box will shrink to absorb that extra width.

And

Here the dimensions of the element are calculated as: width = border + padding + width of the content, and height = border + padding + height of the content.

Now, suppose you include a padding of 45px with the 5px border. In this case both box will be equal but you will notice that the one with border will have a height/width 0 for the content because we already reached 100px with padding and border (45px*2 + 5px*2 = 100px) but the other box will still have some space for content:

div.test {
  background-color: red;
  box-sizing: border-box;
  display: inline-block;
  border: 5px solid;
  text-align: center;
  padding: 45px;
  height:100px;
  width:100px;
  vertical-align:middle;
}

div.test:first-of-type {
  border:none;
}

<div class="test">aa</div>
<div class="test">aa</div>

Now if we start increasing the padding, the first box still have some content to shrink (10px) but the second one no! In this case, border-box and the fixed width/height will have no effect because border + padding exceeded the 100px (46px*2 + 5px*2 = 102px). That's why starting from 45px of padding we see a difference between both boxes and strating from 50px of padding both box will have no content to shrink BUT the second box has more border which will logically make its size bigger. It also become useless to define width or height.

In other words, border-box only applies when padding + border < width/height because only in this case we still have content to shrink.

Here is a better illustration with bigger border and you will see that we have 3 states. (1) when both have content to shrink (2) when only one has content to shrink (3) when both have no content to shrink:

div.test {
  background-color: red;
  box-sizing: border-box;
  display: inline-block;
  vertical-align:top;
  border: 30px solid;
  text-align: center;
  padding: 5px;
  width:100px;
  height:100px;
  animation:pad 10s infinite linear alternate;
}

div.test:first-of-type {
  border:none;
}

@keyframes pad {
 0%  {padding:5px}
 33% {padding:20px}
 66% {padding:50px}
 100% {padding:100px;}
}
.change:after {
  content:"";
  animation:change 10s infinite linear alternate;
}
@keyframes change {
 0%,33%  {content:" (1): same size for both and fixed width/height are respected "}
 33.1%,66% {content:" (2): The second box has no more content to shrink, it starts growing (fixed height/width and border-box has no effect)"}
 66.1%,100% {content:" (3): Both boxes has no more content to shrink, they will keep growing BUT the second one has more border so bigger size"}
}

<p class="change">we are at state </p>
<div class="test">aa</div>
<div class="test">aa</div>

In order to avoid this, we keep the same padding/border for both elements like we initially said.