apache-crunch

How to use Counters in apache crunch


In Apache Crunch , there is method named increment("any enum").

I used increment(TOTAL_IDS);, but where I can see the result of counters, counters are not coming in logs after completion of job.

What am I missing there?


Solution

  • you should be able to see your counters in the tracking URL of the mapreduce job (if you are running mapreduce) or extract the counters from the pipeline. It would be useful if you could provide your code how you are incrementing the counters? is it in your DoFn, in your cleanp method?

    Regards