[Question] How to add custom evaluation metrics? about adanet

[cweill]

Please upgrade your adanet to version >= 0.5.0. You can then pass you custom_metrics to the adanet.Estimator constructor's metric_fn argument: https://adanet.readthedocs.io/en/v0.5.0/adanet.html#estimator

[shawpan]

Thanks. For others the metric_fn should return a tuple of (value, update_op). An example will look like the following metric_fn

def custom_metrics(predictions, features, labels):
    return {
        'customMetric': (tf.constant(0.0), tf.add(tf.constant(0.0) , tf.constant(0.0) ) )
    }

Also the example function my_auc in https://www.tensorflow.org/api_docs/python/tf/contrib/estimator/add_metrics has wrong order of arguments.

Closing this issue.

[cweill]

Generally you want the value element of the tuple to be a variable, and for the update_op to mutate it somehow. For example look at tf.metrics.accuracy.

[shawpan]

Yes, Thanks for more specific and elaborate example tf.metric.accuracy.