---
title: "STAT481/581: Introduction to Time Series Analysis"
author: "Ch1. Getting started"
date: "OTexts.org/fpp3/"
toc: true
colortheme: monashwhite
output:
  binb::monash:
    fig_width: 7
    fig_height: 3.5
    includes:
      in_header: header.tex
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, cache = TRUE)
library(dplyr)
library(ggplot2)
library(fable)
ausbeer <- as_tsibble(fpp2::ausbeer) %>%
  rename(Time = index, Production = value)
austa <- as_tsibble(fpp2::austa) %>%
  rename(Year = index, Visitors = value)
melsyd <- tsibbledata::ansett %>%
  filter(Airports == "MEL-SYD")
```

# What can we forecast?

## Forecasting problems


  1. Google closing stock price on Feb 23rd 2020.
  2. Google closing stock price on March 22nd 2020.
  4. Maximum temperature at Albuquerque airport on Feb 22nd 2020.
  5. The trend estimate of total employment rate for April 2020. 



## Forecasting is difficult

\fullheight{hopecasts2}

## Forecasting is difficult

\fullwidth{bad_forecasts}

## What can we forecast?

\fullwidth{nasdaq-stock-market}

## What can we forecast?

\fullwidth{Forex2}

## What can we forecast?

\fullwidth{pills}

## What can we forecast?

\fullwidth{elecwires2}

## What can we forecast?

\fullheight{AusBOM}


## What can we forecast?

\fullheight{comet}


## Which is easiest to forecast?
\small
 1. daily electricity demand in 3 days time
 2. timing of next Halley's comet appearance
 3. time of sunrise this day next year
 4. Google stock price tomorrow
 5. Google stock price in 6 months time
 6. maximum temperature tomorrow
 7. exchange rate of \$US/AUS next week
 8. total sales of drugs in Australian pharmacies next month

\pause

 - how do we measure "easiest"?
 - what makes something easy/difficult to forecast?

## Factors affecting forecastability

Something is easier to forecast if:

 1. we have a good understanding of the factors that contribute to it
 2. there is lots of data available;
 3. the forecasts cannot affect the thing we are trying to forecast.
 4. there is relatively low natural/unexplainable random variation.
 5. the future is somewhat similar to the past

## Forecasting electricit demand

Can be highly accurate since

 - we have a good idea of the contributing factors: temperatures, small effects for calendar variations, and economic conditions,
 - there is a sufficient history of data on electricity demand and weather conditions.


## Forecasting currency exchange rates

Despite plenty of data, forecasting currency exchange rates can be no more than random guessing since

 - we have a limited understanding of the factors that affect exchange rates,
 - forecasts of the exchange rate have a direct effect on the rates themselves

## Good forecasts

 - Good forecasts capture the genuine patterns and relationships which exist in the historical data, but do not replicate past events that will not occur again
 - In this class, we will learn how to tell the difference between a random fluctuation in the past data that should be ignored, and a genuine pattern that should be modelled and extrapolated.

## Forecasting for decision-making activities

 1. In the early stages of a forecasting project, decide what should be forecast. 
 2. Consider the forecasting horizon: short-term, medium-term and long-term forecasts.
 3. Determine how frequently are forecasts required. Forecasts that need to be produced frequently are better done using an automated system.
 4. Find or collect the data on which the forecasts will be based.
 
 

# Time series data


## Time series data

  - Quarterly Australian beer production

\pause


```{r, echo=FALSE, fig.height=2}
ausbeer %>% autoplot(Production)
```

\pause

**Forecasting is estimating how the sequence of observations will continue into the future.**

## Australian beer production

```{r, echo=FALSE}
ausbeer %>% model(ETS(Production)) %>% forecast() %>% autoplot(ausbeer)
```

## Australian beer production

```{r, echo=FALSE}
ausbeer %>%
  model(ETS(Production)) %>%
  forecast() %>%
  autoplot(tail(ausbeer, 60))
```

## Time series data considered in this course

 - observed sequentially over time
 - observed at regular intervals of time (e.g., hourly, daily, weekly, monthly, quarterly, annually)

# Some case studies

## CASE STUDY 1: Paperware company

\fontsize{12}{14}\sf

\begin{textblock}{7.6}(0.2,1.4)
\textbf{Problem:} Want forecasts of each of hundreds of
items. Series can be stationary, trended or seasonal. They currently
have a large forecasting program written in-house but it doesn't seem
to produce sensible forecasts.

\vspace*{0.1cm}

\textbf{Additional information}\vspace*{-0.2cm}\fontsize{12}{13.5}\sf
\begin{itemize}\itemsep=0cm\parskip=0cm
\item  Program  written in COBOL making numerical calculations limited. 
\item Their programmer has little experience in numerical computing.
\item They employ no statisticians and want the program to produce forecasts \rlap{automatically.}
\end{itemize}
\end{textblock}

\placefig{8}{1.4}{width=4.8cm}{tableware2}

## CASE STUDY 1: Paperware company
### Features of time series data

 - some with trend
 - some seasonal
 - some with no trend nor seasonal

## CASE STUDY 1: Paperware company

### Methods currently used

A
: 12 month average

C
: 6 month average

E
: straight line regression over last 12 months

G
: straight line regression over last 6 months

H
: average slope between last year's and this year's values.
  (Equivalent to differencing at lag 12 and taking mean.)

I
: Same as H except over 6 months.

## CASE STUDY 2: PBS

\fullwidth{pills}


## CASE STUDY 2: PBS

### The Pharmaceutical Benefits Scheme (PBS) is the Australian government drugs subsidy scheme.

  * Many drugs bought from pharmacies are subsidised to allow more equitable access to modern drugs.
  * The cost to government is determined by the number and types of drugs purchased. Currently nearly 1\% of GDP.
  * The total cost is budgeted based on forecasts of drug usage.

## CASE STUDY 2: PBS

\fullheight{pbs2}

## CASE STUDY 2: PBS data
\small

  - In 2001: \$4.5 billion budget, under-forecasted by \$800 million.
  - Thousands of products. Seasonal demand and trend exist.
  - The sales volumes for many groups have sudden jumps up or down due to changes in what drugs are subsidised.
  - The expenditures for many groups also have sudden changes due to cheaper competitor drugs becoming available.
  - Forecasting methods needed to be applied automatically to a large number of time series. 
  

## CASE STUDY 3: Car fleet company

**Client:** One of Australia's largest car fleet companies

**Problem:** how to forecast resale value of vehicles? How
should this affect leasing and sales policies?

\pause

### Additional information
 - They can provide a large amount of data on previous vehicles and their eventual resale values.
 - The resale values are currently estimated by a group of specialists. They see the statistican consultant as a threat and do not cooperate.



## CASE STUDY 4: Airline

\fullwidth{ansettlogo}


## CASE STUDY 4: Airline

```{r, echo=FALSE, fig.height=5}
melsyd %>%
  filter(Class == "Economy") %>%
  autoplot(Passengers) +
  labs(
    title = "Economy class passengers",
    subtitle = "Melbourne-Sydney",
    xlab = "Year"
  ) +
  ylab("Thousands")
```


## CASE STUDY 4: Airline

```{r, echo=FALSE, fig.height=5}
melsyd %>%
  filter(Class == "Economy") %>%
  autoplot(Passengers) +
  labs(
    title = "Economy class passengers",
    subtitle = "Melbourne-Sydney"
  ) +
  ylab("Thousands")
```

\begin{textblock}{4.2}(7,6.3)
\begin{alertblock}{}
Not the real data! Or is it?
\end{alertblock}
\end{textblock}



## CASE STUDY 4: Airline

**Problem:** how to forecast passenger traffic on major routes?

### Additional information

  * They can provide a large amount of data on previous routes.
  * Traffic is affected by school holidays, special events such as
the Grand Prix, advertising campaigns, competition behaviour, etc.
  * They have a highly capable team of people who are able to do
most of the computing.


# The statistical forecasting perspective

## Sample futures

```{r austa1, echo=FALSE, message=FALSE, warning=FALSE, cache=TRUE, fig.width=9, fig.height=6}
fit <- austa %>% model(ETS())
```

```{r austa1a, echo=FALSE, message=FALSE, warning=FALSE, cache=TRUE, fig.width=9, fig.height=6}
sim <- fit %>% generate(h = 10, times = 10) %>%
  mutate(
    replicate = factor(.rep, levels = 1:10, labels = paste("Future", 1:10))
  )
ggplot(austa, aes(x = Year)) +
  geom_line(aes(y = Visitors, colour = "Data")) +
  geom_line(aes(y = .sim, colour = replicate), data = sim) +
  ylab("Millions of visitors") + xlab("Year") +
  ggtitle("Total international visitors to Australia") +
  scale_colour_manual(values = c("#000000", rainbow(10)),
                      breaks = c("Data", paste("Future", 1:10)),
                      name = " ")
```

## Forecast intervals

```{r austa2, echo=FALSE, message=FALSE, warning=FALSE, cache=TRUE, fig.width=8.6, fig.height=6}
fit %>%
  forecast(h = 10) %>%
  autoplot(austa) +
  ylab("Millions of visitors") + xlab("Year") +
  ggtitle("Forecasts of total international visitors to Australia")
```


## Statistical forecasting

\fontsize{14}{16}\sf

- Thing to be forecast: a random variable, $y_t$.
- Forecast distribution: If ${\cal I}$ is all observations, then $y_{t} |{\cal I}$ means ``the random variable $y_{t}$ given what we know in \rlap{${\cal I}$''.}
- The ``point forecast'' is the mean (or median) of $y_{t} |{\cal I}$
- The ``forecast variance'' is $\text{var}[y_{t} |{\cal I}]$
- A prediction interval or ``interval forecast'' is a range of values of $y_t$ with high \rlap{probability.}
- With time series, \rlap{${y}_{t|t-1} = y_t | \{y_1,y_2,\dots,y_{t-1}\}$. }
- $\hat{y}_{T+h|T} =\text{E}[y_{T+h} | y_1,\dots,y_T]$ (an $h$-step forecast taking account of all observations up to time $T$).

## Basic steps in a forecasting task

 1. Problem definition: who, what, how
 2. Gather information: statistical data and accumulated expertise of the people who collect the data and use the forecasts
 3. Preliminary analysis
 4. Choosing and fitting models
 5. Using and evaluating a forecasting model

