tfx

How to compute additional statistics in Evaluator?


In the TFX Evaluator, on top of the metrics described in TFMA format,I would like to compute statistics relative to the performance of my model on my dataset. Naturally, I would also like a way to get access to these statistics: either through the output of the component, or by letting the component upload the statistics somewhere.

I guess that some amount of custom code would be needed (both for the computation and the return of the statistics), but I don't really know how much and what would be the best way to write it. Any ideas on the topic ?

Thanks


Solution

  • There are 2 methods that you can achieve this depending on how you see the placement of your functionality in the TFX flow.

    1. Writing a Custom TFX Components - which requires a lot of effort and you need to define quite a few things.
    2. Reusing Existing Components - instead of writing a component for TFX entirely from scratch, we can inherit an existing component and customize it by overwriting the executor functionality.

    I would suggest the following blogs to begin with:
    Anatomy of TFX Component
    Creating custom Components