"cv_selection_metric" & "cv_report_metrics" about greykite

[sayanpatra]

Hi,

Can you post your code please so that we can reproduce it?

[dromare]

Hi sayanpatra,

I believe this issue can be considered a duplicate of issue #43, which is still open. The basic question is: how can we control the amount of metrics being computed rather than forcibly suppress useful warnings which are there to show us what may be going wrong ?

• The two warnings about y_true (MAPE and medAPE are undefined) most probably refer to the train period, because that is where my response has some zero values (no zero values in the forecast period)
• The warning about y_pred, instead, can either refer to the train period or the forecast period ( because this particular model fits the intercept term, which also becomes the forecast)

I know that my response may contain zeros values, hence I would like to prevent the algorithm from calculating an undefined MAPE rather than letting it go ahead and throw a warning. Similarly, I am not interested in the correlation calculation, so I would like the algorithm to be spared that task and effort too.

[sayanpatra]

When you pass cv_selection_metric as None in EvaluationMetricParam, it computes a default set of metrics. I believe you will not get those error messages if you add the RMSE as the evaluation metric as follows:

from linkedin.greykite.common.evaluation import EvaluationMetricEnum

evaluation_metric = EvaluationMetricParam(
    cv_selection_metric=EvaluationMetricEnum.RootMeanSquaredError.name
)

config = ForecastConfig(
metadata_param=metadata,
...,
evaluation_metric_param=evaluation_metric
)

[dromare]

I did as you said but I am still getting MAPE and MedAPE messages, whether I use the RMSE or the MAE metric (MAE metric in screenshot below)

:

[sayanpatra]

I can not respond based on partial code. As I said before if you need help, post your full code.

[dromare]

I found out that these warnings come from the error metrics on the forecast results (and the backtest results if test_horizon > 0):

result.forecast.train_evaluation
result.forecast.test_evaluation
result.backtest.train_evaluation
result.backtest.test_evaluation