rustvectorbyteslicebiginteger

I can't get the original bytes out of a concatenated byte String in Rust


I've converted a file into bytes and have a vec/array of

[68, 114, 97, 109, 97, 116, 105, 99, 32, 115, 97, 120, 97, 112, 104, 111, 110, 101, 32, 112, 108, 97, 121, 115, 32, 73, 78, 32, 84, 72, 69, 32, 68, 73, 83, 84, 65, 78, 67, 69]

When I concatenated this I ended up with the number 4920555689216983877

Now my issue lies in whenever I try to convert this number back into bytes I end up with only the 8 ending bytes.

I tested this with both Rust's default and the BigInt crate and I'm not satisfied with either:

use num_bigint::{BigUint, ToBigUint};
use num_traits::{Zero, One, ToPrimitive, ToBytes, Num};

fn main() {
    let the_u8: u128 = 4920555689216983877;
    let bigone = (4920555689216983877 as u128).to_biguint().expect("might fail");
    let the_u8bytes: [u8;16] = the_u8.to_be_bytes(); //Using a simple array for demonstrated purposes
    let bigonebytes: Vec<u8> = bigone.to_be_bytes();
    println!("Bytes of base u8: {:?}", the_u8bytes);
    println!("Base of BigInt: {:?}", bigonebytes);
}

[dependencies]
num-bigint = "0.4.4"
num-traits = "0.2.18"

I for some reason can't reassemble the original file which is integral to my program but I feel as though I followed all the logical steps for reverse engineering this simplified process. Could someone explain what's exactly going wrong and help me formulate a strategy to counter it?

I asked a similar question before but didn't have the best minimal reproducible example, now that I have gone through the effort of making a separate project to try to reproduce my issues I'm faced with a similar predicament.

This is the function I used to create the singular int to represent all the bytes in decimal format:

fn concatenate_bytes(bytes: &[u8]) -> BigUint {
    
    println!("Bytes: {:?}", bytes);
    let mut result = 0;

    let mut raw_dog = bytes.as_ptr();
    for _ in bytes.iter() {
        

        unsafe{ 
            result = (result << 8) | (*raw_dog as u64);
            raw_dog = raw_dog.offset(1);
        }
    }
    println!("Result: {}", result);
    result.to_biguint().expect("Failed to create uint")
    //Is actually valid ??? ^
}

Many of the comments were made for myself, but I understand if they aren't particularly helpful for you.


Solution

  • Your concatenate_bytes function is losing information. The result variable is a u64 meaning it will only contain up to 8 bytes of information. If you put a byte into it, and then << 8 eight more times then that byte is "pushed out" of the result. And thus result ends up only reflecting the last 8 bytes.

    If you use a function that preserves all the bytes in a BigUint, like from_bytes_be, then you get a number like this from your original data:

    use num_bigint::BigUint;
    
    let bytes = &[68, 114, 97, 109, 97, 116, 105, 99, 32, 115, 97, 120, 97, 112, 104, 111, 110, 101, 32, 112, 108, 97, 121, 115, 32, 73, 78, 32, 84, 72, 69, 32, 68, 73, 83, 84, 65, 78, 67, 69];
    let biguint = BigUint::from_bytes_be(bytes);
    println!("{biguint}");
    
    571099513965487418103312191225646346334778926396615879607986731592501905638024494155407303787333
    

    Loading this number as a BigUint and using to_bytes_be would return the original bytes.