google-cloud-data-fusioncdap

How to calculate the number of rows in CDAP/DATA Fusion?


How to calculate the number of rows, for example, I use the NullFieldSplitter plugin to divide the data into two parts, and I want to calculate the number of rows for each part. How to calculate it? Someone can take a look and help me, thanks.


Solution

  • You can count number of records using the Group By plugin under the Analytics section. After you split the record, you can pass them to the Group By to calculate the number of records for each of the ports using the Count aggregation method.