Multiple select - value of deselect returns null

Description

I am using a jquery plugin chosen which pretty much does something like this:

This lets me add each option one by one or remove each option one by one. For every option selected, I create a new element with the selected option's value as the id.

Problem

The problem is when I remove an option from the list. For example:

$('#multiple-select').on('change', function(e){
  alert($(e.target).val());
});

Will return the correct value if an option is added, however it returns null if we remove an option. I need to be able to get the value of the option being deselected so I can remove it in the other element.

Question

How can I get the deselected option's value to return the actual value instead of null? Or is there another way to bypass this problem?

All I need to be able to do is find the option being deselected and removing it from the other element (knowing that the element's id is built on the option's value).

Update

Remove code as requested:

$('body').on('change', benefs, function(e){
  var $nbparts = $(participantNbParts),
      $target = $(e.target),
      $val = $target.val(),
      $name = $target.text();

      if($val == null){
        //this is because we deleted something thus we need to remove it from $nbparts which is a pitty since we don't know what it was before it got deleted
      }else{
        //someone was added
        $(create_row_expense_nb_parts_participant($name, $val)).appendTo($nbparts).show('slow');
        $nbparts.parent().show('fast');
      }
});

jQuery chosen provides selected and deselected using which you can identify selected and deselected values respectively, like:

$("#your_element_id").on('change', function (evt, params) {

    var selected = params.selected;
    var removed = params.deselected; //gives you the deselected value

    //assuming your option ids are numbers
    if( removed > 0 ) {
        console.log( "Value removed is:" + removed );
    }

});