typescriptdatabasenestjstypeormnestjs-typeorm

Query Regarding Unexpected Modification of Data Structure in TypeORM Inserts


I'm encountering an unexpected behavior in my shared code repository while working with TypeORM.

Here's a snippet of the code that's causing the issue:

const filteredProducts = //bulky data.
const productChunks = chunkArray(filteredProducts, 100);

for (const productChunk of productChunks) {
    await this.productPriceRepo.insert(productChunk);
    console.log('Dumped in Database ', productChunk.length);
}

products.push(filteredProducts);

The issue arises with the filteredProducts variable. Prior to insertion, its structure differs from after the insertion:

{
  "name":"",
  "category":"",
  "brand":"",
  "properties":{
    "color":"",
    "grade":,
    "carrier":"",
    "storage":""
  },
  "price":,
  "amazonDate":"",
  "country":"",
  "resource":"",
  "asin":"",
  "engine":"",
  "url":"",

  // After insertion, additional fields are added:
  "id": '59c891e',
  "createdAt": ,
  "updatedAt":
}

Furthermore, I have a Base entity class in my codebase, which includes the id, createdAt, and updatedAt fields.

export abstract class Base {
  @PrimaryGeneratedColumn('uuid')
  id: string;

  @CreateDateColumn()
  createdAt: Date;

  @UpdateDateColumn()
  updatedAt: Date;
}

My question is whether this behavior is expected for TypeORM inserts, especially considering that filteredProducts is not directly utilized in the insertion process.

Any insights or guidance would be greatly appreciated.


Solution

  • First of all. Multiple insertions should not be saved like that. You should use transaction. The reason is if some item is not saved correctly in db - the whole transaction should be rolled back. In this case, you avoid inconsistency in your database.

    async createMany(users: User[]) {
      const queryRunner = this.dataSource.createQueryRunner();
    
      await queryRunner.connect();
      await queryRunner.startTransaction();
      try {
       // your for loop in here...  
    
        await queryRunner.commitTransaction();
      } catch (err) {
        // since we have errors lets rollback the changes we made
        await queryRunner.rollbackTransaction();
      } finally {
        // you need to release a queryRunner which was manually instantiated
        await queryRunner.release();
      }
    }
    

    See: Docs of Nestjs

    Just create some generic class for it like below and reuse it

    import { EntityManager, Repository } from 'typeorm';
    
    export async function transactionClosure<T>(
      repo: Repository<any>,
      func: (transactionManager?: EntityManager) => Promise<T>,
    ): Promise<T> {
      const queryRunner = repo.manager.connection.createQueryRunner();
    
      await queryRunner.connect();
      await queryRunner.startTransaction();
    
      try {
        const res: T = await func(queryRunner.manager);
    
        await queryRunner.commitTransaction();
    
        return res;
      } catch (err) {
        await queryRunner.rollbackTransaction();
        throw err;
      } finally {
        await queryRunner.release();
      }
    }