Converting a 16 bit audio file to 24 bit for I2S

I am trying to convert a 16 bit audio samples to 24 bit audio samples file for replaying and mixing purposes. I am shifting it to the left by 8 bytes and then send it to the i2s_channel but the sound im getting is very distorted. This should work, because im doing the sign assignment correct. I am able to get a nice sound when i transmit a 24 bit audio sample, so my configuration should not be the problem. I have an ESP32-S3 for this application

uint8_t* current_pos = (uint8_t*)buf + total_sent_bytes;
size_t num_samples = bytes_to_write / 2;
size_t bytes_to_write = num_samples * 3; // Convert to 24-bit
uint8_t *current_pos_16 = heap_caps_malloc(bytes_to_write, MALLOC_CAP_SPIRAM);
for (int i = 0; i < num_samples; i++) {
    uint32_t sample = (uint32_t)((0x00) |
                      (current_pos[i*2] << 8) |
                      (current_pos[i*2 + 1] << 16));      
    current_pos_16[i*2] = sample & 0xFF; // Padding for 24-bit
    current_pos_16[i*2 + 1] = (sample >> 8) & 0xFF;
    current_pos_16[i*2 + 2] = (sample >> 16) & 0xFF;
}
                    
ESP_ERROR_CHECK(i2s_channel_write(*tx_chan, current_pos_16, bytes_to_write, &written_bytes, 1000));
total_sent_bytes += written_bytes;
free(current_pos_16);

This is how i transmit a 24 bit audio sample that is originally 24 bit :

uint8_t* current_pos = (uint8_t*)buf + total_sent_bytes;                   
uint8_t *current_pos_24 = heap_caps_malloc(bytes_to_write, MALLOC_CAP_SPIRAM);          
                    
for (int i = 0; i < (bytes_to_write / 3); i++) {
      uint32_t sample = (uint32_t)(current_pos[i*3] | 
                        (current_pos[i*3 + 1] << 8) | 
                        (current_pos[i*3 + 2] << 16));      
      current_pos_24[i*3] = sample & 0xFF;
      current_pos_24[i*3 + 1] = (sample >> 8) & 0xFF;
      current_pos_24[i*3 + 2] = (sample >> 16) & 0xFF;
}
ESP_ERROR_CHECK(i2s_channel_write(*tx_chan, current_pos_24, bytes_to_write, &written_bytes, 1000));
total_sent_bytes += written_bytes;
free(current_pos_24);

Why all these unneeded multiplications, divisions and shifts? You are loosing precious resources.

I don't know if there is big or little endian, so here are both.

//assume bytes_to_write is original (x2), not modified to 24 (x3)

//Little endian
for (int i = 0, j=0; i < bytes_to_write; i+=2, j+=3) {
  current_pos_16[j]   = current_pos[i];
  current_pos_16[j+1] = current_pos[i+1];
  current_pos_16[j+2] = 0;
}

//Big endian
for (int i = 0, j=0; i < bytes_to_write; i+=2, j+=3) {
  current_pos_16[j]   = 0;
  current_pos_16[j+1] = current_pos[i];
  current_pos_16[j+2] = current_pos[i+1];
}