During a certain File type conversion from The Raw Tab file That we fetch to convert to Parquet Type, one of the rows is always missing \ in the columns values,
so the expected value for example is :'GH\334' But the Parquet File only has 'GH334' not sure why its skipping the '' especially since we are not doing any sort of data transform, just running a simple Copy Activity.
Design is: Source: CSV file type Sink: Parquet File type Mapping: @json(item().Mapping) (This mapping is provided from a lookup activity which is fetched from the SQL server) no other settings or such, Has anyone dealt with such issues before? would appreciate your inputs on this; Thank you.
Also to mention, this row is being converted to string type for further data transformation...I was wondering if that had any relation with it, But I would need some Ideas on How I can move forward with this Again, the source CSV/.tab file does indeed have the row with '' Character, its just during the copy Activity, that its getting removed
Frankly its such a simple copy Activity, I am sort of out of ideas on what to try, I would appreciate it, if someone could give me an idea on what Can I try
Sample Input Data: GH/DFC\3456
Sample Expected Output Data: GH/DFC\3456
Sample Actual Output Data: GH/DFC3456
@mohammed rifat khan, posting our discussion as an answer for the community.
I took the below data as sample which is similar to your data.
Table2Id,Marks,name,age
1,94,"GH/DFC\3456",25
3,100,"Rakesh/DFC\3456",24
5,98,"GH/Laddu\3456",23
To not skip the \
character, you need to ensure that you have given the \
character as an escape character in the source dataset configurations.
Now, it will give the desired output which you can cross-check from the source dataset data preview.