Homepage
tidyverts / fasster Goto Github PK
View Code? Open in Web Editor NEWForecasting with Additive Switching of Seasonality, Trend and Exogenous Regressors
Home Page: https://fasster.tidyverts.org/
Forecasting with Additive Switching of Seasonality, Trend and Exogenous Regressors
Home Page: https://fasster.tidyverts.org/
Hi!
I'm having some trouble figuring out how to formulate confidence intervals for a model where I am applying a log()
transform to the outcome. The forecast plot looks perfectly reasonable but I need to get my hands on the numerical values of the confidence intervals.
This is what the model looks like:
fit <- data %>%
fasster(
log(y) ~ poly(1) + seas(24)
)
I'm extracting data for the forecast as follows:
fit %>%
forecast(h=24) %>%
pull(forecast) %>%
.[[1]]
and the output looks like this:
time mean se distribution
<dttm> <dbl> <dbl> <dist>
1 2018-09-25 08:00:00 887. 1.31 t(N(6.2, sd = 1.3))
2 2018-09-25 09:00:00 339. 1.34 t(N(5.2, sd = 1.3))
3 2018-09-25 10:00:00 399. 1.38 t(N(5.3, sd = 1.4))
4 2018-09-25 11:00:00 367. 1.43 t(N(5.2, sd = 1.4))
5 2018-09-25 12:00:00 40.4 1.47 t(N(3, sd = 1.5))
6 2018-09-25 13:00:00 103. 1.51 t(N(3.9, sd = 1.5))
7 2018-09-25 14:00:00 580. 1.55 t(N(5.6, sd = 1.5))
8 2018-09-25 15:00:00 858. 1.59 t(N(5.9, sd = 1.6))
9 2018-09-25 16:00:00 2981. 1.62 t(N(7.2, sd = 1.6))
10 2018-09-25 17:00:00 3459. 1.66 t(N(7.3, sd = 1.7))
Am I right in assuming that the 95% confidence interval for the first forecast point would be
qlnorm(c(0.025, 0.975), 6.2, 1.3)
If there's a function for extracting the confidence intervals that would be great. However, I have no problem extracting them by hand. Just want to know that I'm doing the right thing!
Thanks,
Andrew.
Could you please have a look at ?tsibble::reexports
? If there are lots of verbs you used that have been re-exported, you may like to remove them as explicit dependencies.
First of, the package is really great, thanks a lot ! I have already used msts
to define the time series (passenger_count) but I get the following warning of not using time series,
fasster:::fasster(numberOfPassengersTimeseries, passenger_count ~ temperature + Rain, heuristic = c("filterSmooth")) %>% ggfitted
Warning message:
In fasster:::fasster(numberOfPassengersTimeseries, passenger_count ~ :
Time series information not found, please provide a tsibble or time-series object.
I also add the coefficients of AR and MA as vectors but I get error. Can you please give an example of a arma added to fasster?
Thanks in advance
Potential UI:
FASSTER(y ~ poly(1) + G %S% (trig(12)) + (G == "A") %only% (x + z))
I just followed the code example on the README but when I tried to run the forecasting function, I got an error message "Error: x is not a tsibble". Any ideas? Thanks in advance!
library(fable)
fit <- as_tsibble(USAccDeaths) %>%
model(fasster = FASSTER(value ~ poly(1) + trig(12)))
fit %>%
forecast(h=24) %>%
autoplot(as_tsibble(USAccDeaths))
Hi, Great work on the package by the way.
I want to generate my formula dynamically from a list of variable names. So I tried to paste them into a string and convert that into a formula. However, this is throwing an error saying "No supported inverse for this function"
Example:
# Preparing the data
airquality %>% mutate(
date = as.Date(paste('2018', Month, Day, sep = '-')),
Ozone = round(imputeTS::na.interpolation(Ozone, option = 'stine')),
Solar.R = round(imputeTS::na.interpolation(Solar.R, option = 'stine'))
) %>%
as_tsibble()
# Creating formula
f <- as.formula('y ~ Wind + Temp + Solar.R + trig(12) + poly(1) + ARMA(1, 1)')
# Building fasster without formula parsing
fsModel <- FASSTER(data = train,
formula = y ~ Wind + Temp + Solar.R + trig(12) + poly(1) + ARMA(1, 1))
# Building fasster - throws an error
fsModel <- FASSTER(data = train, formula = f)
I am not sure if I am doing anything wrong here. The same formula when passed directly worked perfectly fine.
fit <- USAccDeaths %>%
as_tsibble %>%
model(fasster = FASSTER(value ~ poly(1) + trig(12)))
gives an error
Error in .subset2(public_bind_env, "initialize")(...) : argument ".env" is missing, with no default
Hi Mitchell,
Would you please check the forecast function?
library(fasster)
library(tsibble)
library(tidyverse)
ped_sthx <- pedestrian %>%
filter(
Sensor == "Southern Cross Station",
yearmonth(Date_Time) < yearmonth("2015 Apr")
)
library(lubridate)
fit <- ped_sthx %>%
model(
mdl = fasster(Count ~ poly(1) + (trig("week", 8)))
)
fit %>% forecast(h = 10)
Error in FUN(left, right) : non-numeric argument to binary operator
In addition: Warning message:
In mean.default(x, na.rm = TRUE) :
argument is not numeric or logical: returning NA
Another example directly from the forecast.TSLM {fable} R documentation
library(fable)
as_tsibble(USAccDeaths) %>%
model(lm = TSLM(log(value) ~ trend() + season())) %>%
forecast()
It also doesn't work.
Error in as.data.frame.default(x[[i]], optional = TRUE, stringsAsFactors = stringsAsFactors) :
cannot coerce class ‘"mdl_ts"’ to a data.frame
In addition: Warning message:
In mean.default(x, na.rm = TRUE) :
argument is not numeric or logical: returning NA
Could you please help me with this?
Thank you for checking this post!
Such as allowing the include
option to apply only to the heuristic estimation step.
When computing specials, if some elements never switch in then the resulting specials may be of incorrect dimension.
Ideally, improving speed, functionality, and object structures.
To avoid conflicts with potentially useful xreg function stats::poly()
I have a tsibble from this competion https://www.kaggle.com/c/demand-forecasting-kernels-only/overview, fasster is truly fast and cool, but I think I can get 4x performance if I can fit the model on 4 cores.
Evaluate accuracy
I was trying to reproduce your example and notice one small edit that may help future users :)
library(lubridate)
library(forecast)
tbl_taylor <- as_tsibble(taylor) %>%
mutate(index = seq(ymd_h("2000-6-5 00"), by="30 mins", length.out=length(taylor)),
DayType = ifelse(wday(index) %in% 2:6, "Weekday", "Weekend")) %>%
filter(month(index) == 8)
tbl_taylor %>%
filter(index < ymd("2000-8-21")) %>%
# original line
# fasster(taylor~ DayType %S% (poly(1) + trig(48, 16))) %>%
# revised line
fasster(value ~ DayType %S% (poly(1) + trig(48, 16))) %>%
forecast(newdata=tbl_taylor %>%
filter(index >= ymd("2000-8-21"))) %>%
autoplot() +
ylab("Electricity Demand (Megawatts)") +
xlab("Time (5 June 2000 - 27 August 2000)")
When using a switching model, both stream() and forecast() parse the new_data in a way that is inconsistent with the already trained model.
Consider the following model:
elec_tr <- tsibbledata::vic_elec %>%
filter(
yearmonth(Time) == yearmonth("2012 Mar")
) %>%
mutate(WorkDay = wday(Time) %in% 2:6 & !Holiday)
elec_fit <- elec_tr %>%
model(
fasster = fasster(Demand ~
WorkDay %S% (trig(48, 16) + poly(1)) + Temperature + I(Temperature^2)
)
)
elec_fit
It is trained on the boolean switch WorkDay, however if the new_data input is too short, like in this case:
elec_update <- tsibbledata::vic_elec %>%
filter(
yearmonth(Time) == yearmonth("2012 Apr")
) %>%
head(2) %>%
mutate(WorkDay = wday(Time) %in% 2:6 & !Holiday)
elec_fit_upd <- elec_fit %>% stream(elec_update)
elec_fit_upd
then internal DLM states and X will have wrong dimensions as they don't account for unseen states (say, holidays/weekends when the data contains only weekdays):
Error in rbind(object$dlm$X, X) :
number of columns of matrices must match (see arg 2)
Calls: <Anonymous> ... .f -> stream.mdl_ts -> stream -> stream.FASSTER -> rbind
Just to try out the streaming functionality, I fixed it by patching stream() to always use longer new_data, but then tail()'ing X/est/residuals when needed. Obviously, this is extremely ugly.
Not sure how to fix it at the moment. Am I missing something? If you have any ideas to share, I'd love to try fixing it and submitting a PR.
Thanks!
Potential directions include loess and sampling techniques.
Automatic model specification should also be considered with this.
fit <- USAccDeaths %>%
as_tsibble() %>%
model(fasster = FASSTER(value ~ poly(1) + trig(12)))
fit %>%
forecast(h=24)
gives Error in FUN(left, right) : non-numeric argument to binary operator
Can state-switching only handle 2 levels? {0,1}
I had tried with three, but the forecast resulted in a flat line (y_hat=0)
No need to parse this into a dlm, treat it as data
With associated methods for print, residuals, and other typical methods
Especially the fortify method, should aggregate the like terms appropriately.
Hello, I appreciate this is a very new bug which has come with a recent update in the fablelite
package but I thought I would flag. I'm really enjoying exploring the capabilities of this package so hopefully this can be resolved soon.
model.R has a bug where
dlmModel <- model_inputs$args %>%
unlist(recursive = FALSE) %>%
reduce(`+`)
should be
dlmModel <- model_inputs$specials %>%
unlist(recursive = FALSE) %>%
reduce(`+`)
this is also the case for other places where parse_model is used. I can't seem to push my fixed code which is why this is an issue and not a pull request.
As per subject; taking out poly()
term avoids the error.
library(fasster)
library(tsibble)
library(lubridate)
daytype <- function(date){
factor(ifelse(wday(date) %in% 2:6, "Weekday", "Weekend"), levels = c("Weekday", "Weekend"))
}
SthCross_Ex <- pedestrian %>%
filter(Sensor == "Southern Cross Station") %>%
mutate(DayType = daytype(Date)) %>%
as_tsibble()
SthCross_fasster_fit <- SthCross_Ex %>%
fasster(Count ~ DayType %S% (poly(1) + trig(24)))
#> Error in dlmSmooth.dlmFiltered(filtered): error code 1 from Lapack routine dgesdd
SthCross_fasster_fit <- SthCross_Ex %>%
fasster(Count ~ DayType %S% (poly(1)))
Created on 2018-07-29 by the reprex package (v0.2.0).
Does Fasster perform similarly to the forecast
package in that it can handle transformations in model and then back transform at forecasting, such as taking the difference or a seasonal decomposition?
Copied comments of 63f53d7 for more visibility.
Hi Mitchell, I have been your R package 'fable' and 'FASSTER' fan for a while. After FASSTER changes the way it incorporates newdata to the forecast function, my original code couldn't work.
Can you provide an example of how to include new_data into the forecast function (like how you do it to forecast the Australian electricity demand)?
The newdata I need to provide is "daytype" before the switching term %S% ( exactly like what you did for the Australian electricity forecast). I read your code and relevant information online regarding this change but still couldn't figure out how to do it by myself.
Appreciated,
Yijun (@yijunwang0805)
If no formula is specified, a model can be automatically identified.
There are some complexities with this:
Hi,
I viewed your presentation from useR2018 and was deeply impressed by what's possible with fasster
. This looks really amazing!
I've been working my way through the examples of the README
.
> fit <- tsibbledata::UKLungDeaths %>%
+ FASSTER(fdeaths ~ mdeaths)
>
> class(fit)
[1] "mable" "lst_ts" "tbl_df" "tbl" "data.frame"
>
> fit %>% summary
FASSTER Model:
fdeaths ~ mdeaths
Estimated variances:
State noise variances (W):
mdeaths
1.7119e-34
Observation noise variance (V):
1.6631e+03
Everything up to this point works perfectly. But when I try
fit %>% components
or
fit %>% forecast(h=24) %>% autoplot
I get an error:
Error in .f(.x[[i]], ...) : object 'object' not found
Please let me know if you need more information.
Best regards,
Andrew.
Here's my session information:
> devtools::session_info()
Session info ------------------------------------------------------------------------------------------------------------------------------------------------
setting value
version R version 3.5.1 (2018-07-02)
system x86_64, linux-gnu
ui RStudio (1.1.456)
language en_ZA:en
collate en_ZA.UTF-8
tz Africa/Johannesburg
date 2018-09-14
Packages ----------------------------------------------------------------------------------------------------------------------------------------------------
package * version date source
assertthat 0.2.0 2017-04-11 CRAN (R 3.5.1)
base * 3.5.1 2018-07-03 local
bindr 0.1.1 2018-03-13 CRAN (R 3.5.1)
bindrcpp * 0.2.2 2018-03-29 CRAN (R 3.5.1)
colorspace 1.3-2 2016-12-14 CRAN (R 3.5.1)
compiler 3.5.1 2018-07-03 local
curl 3.2 2018-03-28 CRAN (R 3.5.1)
datasets * 3.5.1 2018-07-03 local
devtools 1.13.6 2018-06-27 CRAN (R 3.5.1)
digest 0.6.17 2018-09-12 cran (@0.6.17)
dlm 1.1-5 2018-06-13 cran (@1.1-5)
dplyr 0.7.6 2018-06-29 CRAN (R 3.5.1)
fable * 0.0.0.9000 2018-09-14 Github (tidyverts/fable@6e81dda)
fablelite * 0.0.0.9000 2018-09-14 Github (tidyverts/fablelite@98271e9)
fasster * 0.1.0.9000 2018-09-14 Github (tidyverts/fasster@7633a99)
forecast 8.4 2018-06-21 CRAN (R 3.5.1)
fracdiff 1.4-2 2012-12-02 CRAN (R 3.5.1)
ggplot2 3.0.0.9000 2018-09-14 Github (tidyverse/ggplot2@1e01e68)
glue 1.2.0 2017-10-29 CRAN (R 3.5.1)
graphics * 3.5.1 2018-07-03 local
grDevices * 3.5.1 2018-07-03 local
grid 3.5.1 2018-07-03 local
gtable 0.2.0 2016-02-26 CRAN (R 3.5.1)
lattice 0.20-35 2017-03-25 CRAN (R 3.5.0)
lazyeval 0.2.1 2017-10-29 CRAN (R 3.5.1)
lmtest 0.9-36 2018-04-04 CRAN (R 3.5.1)
lubridate 1.7.4 2018-04-11 CRAN (R 3.5.1)
magrittr 1.5 2014-11-22 CRAN (R 3.5.1)
memoise 1.1.0 2017-04-21 CRAN (R 3.5.1)
methods * 3.5.1 2018-07-03 local
munsell 0.5.0 2018-06-12 CRAN (R 3.5.1)
nlme 3.1-137 2018-04-07 CRAN (R 3.5.0)
nnet 7.3-12 2016-02-02 CRAN (R 3.5.0)
numDeriv 2016.8-1 2016-08-27 CRAN (R 3.5.1)
parallel 3.5.1 2018-07-03 local
pillar 1.2.3 2018-05-25 CRAN (R 3.5.1)
pkgconfig 2.0.1 2017-03-21 CRAN (R 3.5.1)
plyr 1.8.4 2016-06-08 CRAN (R 3.5.1)
purrr 0.2.5 2018-05-29 CRAN (R 3.5.1)
quadprog 1.5-5 2013-04-17 CRAN (R 3.5.1)
quantmod 0.4-13 2018-04-13 CRAN (R 3.5.1)
R6 2.2.2 2017-06-17 CRAN (R 3.5.1)
Rcpp 0.12.18 2018-07-23 cran (@0.12.18)
rlang 0.2.2 2018-08-16 cran (@0.2.2)
scales 1.0.0 2018-08-09 CRAN (R 3.5.1)
stats * 3.5.1 2018-07-03 local
stringi 1.2.4 2018-07-20 CRAN (R 3.5.1)
stringr 1.3.1 2018-05-10 CRAN (R 3.5.1)
tibble 1.4.2 2018-01-22 CRAN (R 3.5.1)
tidyr 0.8.1 2018-05-18 CRAN (R 3.5.1)
tidyselect 0.2.4 2018-02-26 CRAN (R 3.5.1)
timeDate 3043.102 2018-02-21 CRAN (R 3.5.1)
tools 3.5.1 2018-07-03 local
tseries 0.10-45 2018-06-04 CRAN (R 3.5.1)
tsibble 0.5.2.9000 2018-09-14 Github (tidyverts/tsibble@0bc8ceb)
tsibbledata 0.0.0.9000 2018-09-14 Github (tidyverts/tsibbledata@3c2baef)
TTR 0.23-3 2018-01-24 CRAN (R 3.5.1)
urca 1.3-0 2016-09-06 CRAN (R 3.5.1)
uroot 2.0-9 2017-01-29 CRAN (R 3.5.1)
utils * 3.5.1 2018-07-03 local
withr 2.1.2 2018-03-15 CRAN (R 3.5.1)
xts 0.11-1 2018-09-12 CRAN (R 3.5.1)
yaml 2.2.0 2018-07-25 CRAN (R 3.5.1)
zoo 1.8-3 2018-07-16 CRAN (R 3.5.1)
MRE:
fasster(USAccDeaths * 1000 ~ poly(1) + trig(12))
I'm interested to try this package. But I can't reproduce the examples from the README. It seems it's due to the changed dataset from tsibbledata
and maybe also some api changes. Could you update the tutorial? And I tried the following example. I got the error like "Error in attr(data, "tsp") <- c(start, end, frequency) : object is not a matrix". Any thoughts?
library(fasster)
library(fable)
library(tidyverse)
library(forecast)
tbl_taylor <- as_tsibble(taylor) %>%
mutate(index = seq(ymd_h("2000-6-5 00"), by="30 mins", length.out=length(taylor)),
DayType = ifelse(wday(index) %in% 2:6, "Weekday", "Weekend")) %>%
filter(month(index) == 8)
tbl_taylor %>%
filter(index < ymd("2000-8-21")) %>%
fasster(taylor ~ DayType %S% (poly(1) + trig(48, 16))) %>%
fablelite::forecast(h=40) %>%
autoplot() +
ylab("Electricity Demand (Megawatts)") +
xlab("Time (5 June 2000 - 27 August 2000)")
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.