allennlp

AllenNLP - configuration to write test set metrics to Tensorboard

When running the AllenNLP train or evaluate CLI commands, is there a configuration option (in the json/jsonnet file) to write test set evaluation metrics to Tensorboard?

If not, how can I do it in my own script?

Thanks in advance for your time and help. Best regards

Solution

  • You could pass in your test set as a validation set in your config file. However, treating the test set as an evaluation set is not recommended.

    You can also use the allennlp evaluate command, which dumps the metrics to a user-specified output file.