mcompetitions / m5-methods Goto Github PK

Just curious, how long did it take to run the benchmarking code (assuming it's running in parallel)?

I have been trying to run the code on my Mac Desktop(12cores, 30G Memory) but it's been running almost a day.

there is no field id in sales_train_evaluation.csv located on google drive

iMAPA does not return forecast. Should we add return function at the end of iMAPA function ?

iMAPA <- function(x, h){
  mal <- round(mean(intervals(x)),0)
  frc <- NULL
  for (al in 1:mal){
    frc <- rbind(frc, rep(SexpS(as.numeric(na.omit(as.numeric(rollapply(tail(x, (length(x) %/% al)*al), al, FUN=sum, by = al)))), 1)/al, h))
  }
  forecast <- colMeans(frc)
  return(forecast)

}

I used the dataset from Kaggle and tried to reproduce the benchmark results. It took quite a while to run through the forecasts. But it seems to fail when fitting the "auto.arima" model with external variables. Any ideas?

arimax_f <- forecast(auto.arima(insample_top, xreg=as.matrix(head(x_var, length(insample_top)))), h=28, xreg=as.matrix(tail(x_var, 28)))$mean #ARIMA with external variables
Error in auto.arima(insample_top, xreg = as.matrix(head(x_var, length(insample_top)))) :
No suitable ARIMA model found

Hi,

I have a question about the rounding of the statistical benchmarks.
For example I see that the forecast in the first series for ES_bu is:

[1] 0.9829610 1.1039467 0.9006081 0.9184879 1.0185274 1.3266206
[7] 1.0978010 0.9829610 1.1039467 0.9006081 0.9184879 1.0185274
[13] 1.3266206 1.0978010 0.9829610 1.1039467 0.9006081 0.9184879
[19] 1.0185274 1.3266206 1.0978010 0.9829610 1.1039467 0.9006081
[25] 0.9184879 1.0185274 1.3266206 1.0978010

However, I am not able to find, in the Point Forecasts - Benchmarks.R file, any places where the forecast will be rounded.
I am assuming that in a business context (since it is WalMart data) that we have to round the numbers as we cannot give fractional numbers for planning purposes?

there is no column d in data on google drive

https://drive.google.com/drive/folders/1D6EWdVSaOtrP1LEFh1REjI3vej6iUS_4?usp=sharing

Hi!

I'm trying to clone the repo to reproduce the results; I'm using git lfs clone but I get the following error.
Any suggestions on how to fix it?

may you tell us, when winning code will be published in this repo or other place?

as it is written in https://mofc.unic.ac.cy/m5-competition/
Reproducibility
The prerequisite for winning any prize will be that the code used for generating the forecasts, with the exception of companies providing forecasting services and those claiming proprietary software, will be put on GitHub, not later than 14 days after the end of the competition (i.e., the 14th of July, 2020).

Hi, when I run the code, the following error appears;
Error in starting_period:nrow(sales_train): NA/NaN argument
Warning message in eval(expr, envir, enclos):
“NAs introduced by coercion”

I think this is the line that has problems:
sales_train <- as.numeric(ex_sales[,6:ncol(ex_sales)])

I changed it to:
sales_train <- as.numeric(ex_sales[,7:ncol(ex_sales)])

But I'm not sure if it's ok to make this change.

Thanks and Regards,
John Taco

Hi there, first of all, much thanks for all the effort and insights resulting from this competition
(now deep diving on findings paper). Amazing work and contribution!

I was looking for one thing couldn't find so far, would be possible to know or have an idea of compute and time needed by the benchmarks and winning submissions? In practice, it's a relevant dimension for evaluating different approaches.

Example: if I understood properly for exp smooth bottom up, fit was run ~30k times? (number of time series at maximum level). From the code done in parallel I think, but still, prob takes some time.

Would be great to get any info on this.

thanks!
(from https://github.com/Mcompetitions/M5-methods/blob/60829cf13c8688b164a7a2fc8c4832cc216bdbec/validation/Point%20Forecasts%20-%20Benchmarks.R)

Hello,

I'm working on the U3 submission and I have some questions about the uncertainty prediction part.

In the documentation he said:

"To get predictions for all 9 quantiles (median + 8 other ) we can simply multiply every median to a coefficient ; this coefficient was calculated per level by minimizing loss over the last 28 known days (public LB)."

But I didn't find any function relating to minimisation in his scripts; there are only the raw coefficients.
Does anyone know how did he makes the minization? Did he learn thanks to the public LB evaluation metric?
I also wonder why in some of his get_ratio() functions he go through log(qs/(1-qs)) with qs = [0.005,0.025,0.165,0.25, 0.5, 0.75, 0.835, 0.975, 0.995]

Thanks in advance :)

Hi All,

I was just looking to run some of the benchmarks outlined in the competitor guide.

But the 'Point Forecasts - Benchmarks.R' and 'Probabilistic Forecasts - Benchmarks.R' files in the validation folder look to be empty?

Will these be uploaded anytime soon? Would be really helpful if so!

Best,

Shaheen