Ajax File Upload in Codeigniter

there is multiple file upload widgets on my view page, but for reference i have only added 1st here, what i want to do is, i want to select the file using the button and then by ajax method i want to upload the file to directory and get the file name back into input with name "passport_copy_upload" field.

like this

all other file upload widget on this page will work and at the end we will submit the form and take user to thank you page.

Problem is : File Not getting uploaded to directory , in Network Console getting error : {"error":"You did not select a file to upload."}

My view part

        <form id="fimilyVisaForm" action="https://www.mydomainurl.com/visa/add-other-details/<?php echo $appid;?>" method="post" enctype="multipart/form-data" onsubmit="return(validate());">

                                <h4 class="row marginBottom">Upload Documents</h4>

                <div class="form-control">
                    <div class="col-sm-4 label">Colored Scanned copy of the Passport</div>
                    <div class="col-sm-3">
                       <input type="text" class="form-control-input" placeholder="Colored Scanned copy of the Passport" name="passport_copy_upload" id="passport_copy_upload" onclick="removeError(this.id);" value="">
                    </div>   
                    <div class="col-sm-3">   
                       <input type="button" class="submit-btn read-more" value="Attach" id="passport">
                       <input type="file" name="passportimg" id="passportimg" style="display:none">
                       <a id="viewct" class="submit-btn read-more" href="#">View</a>
                      <div class="labelInfo">Colored Scanned copy of the Passport</div>
                    </div>
                </div>
    </form>

<script>
  document.getElementById('passport').onclick = function() {
    document.getElementById('passportimg').click();
};
</script>

Javascript Ajax Code

<script>
$("#passportimg").change(function() {
    var file_data = $("#passportimg").prop("files")[0];
    var form_data = new FormData();
    form_data.append("file", file_data);
    $.ajax({
        url:'<?= base_url();?>index/do_upload',
        dataType: 'json',
        cache: false,
        async: false,
        contentType: false,
        processData: false,
        data: form_data,                         
        type: 'post',
        success:function(data)  
                     {  
                          console.log(data);
                     } 
    });
});
</script>

This is my controller

    public function do_upload() { 
      header('Content-Type: application/json');

      $config['upload_path']   = '<?= base_url();?>uploads/applicant/'; 
      $config['allowed_types'] = 'gif|jpg|png|jpeg'; 
      $config['max_size']      = 2048;
      $this->load->library('upload', $config);
          $this->upload->initialize($config);

      if ( ! $this->upload->do_upload('passportimg')) {
         $error = array('error' => $this->upload->display_errors()); 
         echo json_encode($error);
      }else { 
         $data = $this->upload->data();
         $success = ['success'=>$data['file_name']];
         echo json_encode($success);
      } 
   } 

Kindly replace:

var file_data = $("#passportimg").prop("files")[0];
var form_data = new FormData();
form_data.append("file", file_data);

To:

var form = $("#fimilyVisaForm")[0];
var form_data = new FormData(form);