---
title: "Time Series Homework: Chapter 6"
subtitle: "Please_put_your_name_here"
format: 
  html:
    embed-resources: true
    toc: true
---

```{r, echo=FALSE, message=FALSE, warning=FALSE}

# Loading R packages

if (!require("pacman")) install.packages("pacman")
pacman::p_load(nlme,tsibble, fable,feasts, tsibbledata,
               fable.prophet, patchwork,lubridate, 
               rio,ggplot2,kableExtra,tidyquant, 
               dplyr , plotly,tidymodels, stats,
               multilevelmod, broom.mixed,
               forecast
)
```

## Data

```{r, message=FALSE}

gdp_ts <- rio::import("https://byuistats.github.io/timeseries/data/gdp_fred.csv") |>
  select(-comments) |>
  mutate(year_over_year = gdp_millions / lag(gdp_millions, 4)-1) |>
  mutate(quarter = yearquarter(mdy(quarter))) |>
  filter(quarter >= yearquarter(my("Jan 1990")) & quarter < yearquarter(my("Jan 2025"))) |>
  na.omit() |>
  mutate(t = 1:n()) |>
  mutate(std_t = (t - mean(t)) / sd(t)) |>
  as_tsibble(index = quarter)

```

## Questions

### Question 1 - Key Definitions (10 points)

Answer the prompt to the learning outcome below. Include any mathematical expressions or illustrations that may accompany the definitions and ideas if available.

::: {.callout-note title="Answer" icon="false"}
-   **Define a moving average (MA) process** <!-- Delete this and put your answer here. -->

-   **Write an MA(q) model in terms of the backward shift operator** <!-- Delete this and put your answer here. -->

-   **State the mean and variance of an MA(q) process** <!-- Delete this and put your answer here. -->

-   **Write the equation for an ARMA(p,q) model** <!-- Delete this and put your answer here. -->

-   **Express an ARMA model in terms of the backward shift operators for the AR and MA components** <!-- Delete this and put your answer here. -->
:::

### Question 2 - Stationary Model Comparison (20 points)

We have worked with AR(p), MA(q), ARMA(p,q) stochastic processes. The theoretical ACF and PACF functions of each can help us identify which of the three models would be a good fit for a particular time series. From the Time Series Notebook:

::: {.callout-note icon="false" title="ACF and PACF of an $AR(p)$ Process"}
|      | AR(p)                  | MA(q)                  | ARMA(p,q) |
|------|------------------------|------------------------|-----------|
| ACF  | Tails off              | Cuts off after lag $q$ | Tails off |
| PACF | Cuts off after lag $p$ | Tails off              | Tails off |
:::

The table above summarizes the theoretical functions. Each time series is a realization of the stochastic process, so our sample estimates of these functions might be different. The following questions will walk you through the simulation of each of them to illustrate this point.

#### AR(3) Simulation

```{r}
n <- 10000  # Number of observations
p <- c(0.5, 0.3, 0.1)  # AR coefficients

# Simulate the AR(3) process
set.seed(123)  # Setting seed for reproducibility
ar3_process <- arima.sim(n = n, model = list(ar = p), sd = 1)
ar3_df <- tibble(time = 1:n, value = ar3_process)# Convert the df to a tibble
ar_3 <- as_tsibble(ar3_df, index = time) # Convert the tibble to a tsibble
```

#### MA(3) Simulation

```{r}
n <- 10000  # Number of observations
q <- c(-0.7, -0.5, 0.2)  # MA coefficients

# Simulate the MA(3) process
set.seed(123)  # For reproducibility
ma3_process <- arima.sim(n = n, model = list(ma = q), sd = 1)
ma_3 <- as_tsibble(tibble(Time = 1:n, value = ma3_process), index = Time) # Convert the simulated MA(3) process to a tsibble

```

#### ARMA(3,3) Simulation

```{r}
n <- 10000  # Number of observations
ar_coeffs <- c(0.5, 0.3, 0.1)   # AR coefficients
ma_coeffs <- c(-0.7, -0.5, 0.2)  # MA coefficients

# Simulate the ARMA(3, 3) process
set.seed(123)  # For reproducibility
arma33_process <- arima.sim(n = n, model = list(ar = ar_coeffs, ma = ma_coeffs), sd = 1)

# Convert the simulated ARMA(3, 3) process to a tsibble
arma_33 <- as_tsibble(tibble(Time = 1:n, value = arma33_process), index = Time)
```

##### a) Plot the ACFs for the AR(3), MA(3) and ARMA(3,3) simulations of the same plot.

::: {.callout-note title="Answer" icon="false"}
```{r}


```
:::

##### b) Plot the PACFs for the AR(3), MA(3) and ARMA(3,3) simulations of the same plot.

::: {.callout-note title="Answer" icon="false"}
```{r}



```
:::

##### c) Does the theoretical description on this question prompt match with the plots you created? Please change the seed for the simulation multiple times before answering this question.

::: {.callout-note title="Answer" icon="false"}
<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

### Question 3 - ARMA(p,q): GDP Business Cycles (75 points)

Business cycles are the fluctuations in economic activity that an economy experiences over time, marked by phases of expansion, peak, contraction (or recession), and trough. During expansion, economic indicators such as GDP rise, reaching a peak where the economy operates at maximum capacity. Following the peak, the economy contracts, leading to a recession characterized by falling GDP, until it hits a trough, the lowest point, signaling the end of contraction and the start of recovery. The yearly percentage change in real GDP effectively captures the fluctuations of business cycles by quantifying the rate at which an economy's total output of goods and services grows or shrinks from one year to the next. The yearly percentage change in real US GDP in decimals is the `year_over_year` variable in the `gdp_ts` tsibble.

##### a) Plot the yearly percentage change in US Real GDP

::: {.callout-note title="Answer" icon="false"}
```{r}



```
:::

##### b) Plot the ACF and PACF of the series

::: {.callout-note title="Answer" icon="false"}
```{r}



```
:::

##### c) Based on the ACF and PACF, please speculate which stationary model will best fit the data.

::: {.callout-note title="Answer" icon="false"}
<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

##### d) Please fit an AR(p) model to the yearly percentage change in US Real GDP.

::: {.callout-note title="Answer" icon="false"}
```{r}



```

<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

##### e) Please fit an MA(q) model to the yearly percentage change in US Real GDP.

::: {.callout-note title="Answer" icon="false"}
```{r}



```

<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

##### f) Please fit an ARMA(p,q) model to the yearly percentage change in US Real GDP.

::: {.callout-note title="Answer" icon="false"}
```{r}



```

<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

##### g) In a table, please compare the fit of each model. Which do you think is the best model for the series?

::: {.callout-note title="Answer" icon="false"}
```{r}



```

<!-- Delete this and put your answer here. The three colons mark the end of your answer. -->
:::

### Rubric

+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| **Criteria**                                              |                                                                                                                                                                                                                                                                                                                                                                                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                   |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (10)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 1: Definitions**                          | The student correctly defined each of the terms and included mathematical expressions or illustration if available in the text or the Time Series Notebook                                                                                                                                                                                                                                                                                             | The student did not provide a correct definition for one or more of the terms.                                                                                                                                                                                                                                                                                                                    |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (5)**                                                                                                                                                                                                                                                                                                                                                                                                                                        | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 2a: ACF Plots**                           | Students produce clear and well-labeled plots of the autocorrelation functions (ACFs) for the AR(3), MA(3), and ARMA(3,3) simulations.                                                                                                                                                                                                                                                                                                                 | Submissions have low-quality visualizations or unclear labeling.                                                                                                                                                                                                                                                                                                                                  |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (5)**                                                                                                                                                                                                                                                                                                                                                                                                                                        | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 2b: PACF**                                | Students produce clear and well-labeled plots of the partial autocorrelation functions (PACFs) for the AR(3), MA(3), and ARMA(3,3) simulations.                                                                                                                                                                                                                                                                                                        | Submissions have low-quality visualizations or unclear labeling.                                                                                                                                                                                                                                                                                                                                  |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (10)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 2c: Evaluation of Theoretical functions** | Students contrast the theoretical description provided in the question prompt and compare it with the plots they created from multiple simulations with different seeds. They accurately identify and explain any discrepancies or agreements between the theoretical description and the observed patterns in the plots. Their analysis demonstrates a clear understanding of sampling distributions and their representations on ACF and PACF plots. | Students contrast doesn't provide sufficient evidence of understanding of simulation procedures or the theoretical understanding of ACF and PACF. There is not enough content to evaluation whether students understand the sampling process and estimation of ACF and PACF.                                                                                                                      |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (5)**                                                                                                                                                                                                                                                                                                                                                                                                                                        | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3a: Plot**                                | Students plot the yearly percentage change in US Real GDP, ensuring that the plot is clear, properly labeled, and accurately represents the data. They provide appropriate axis labels, a title, and a legend if necessary to enhance interpretability.                                                                                                                                                                                                | Submissions have low-quality visualizations or unclear labeling.                                                                                                                                                                                                                                                                                                                                  |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (5)**                                                                                                                                                                                                                                                                                                                                                                                                                                        | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3b: ACF and PACF**                        | Students produce clear and well-labeled plots of the ACF and PACF                                                                                                                                                                                                                                                                                                                                                                                      | Submissions have low-quality visualizations or unclear labeling.                                                                                                                                                                                                                                                                                                                                  |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (15)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3c: Best fit speculation**                | Students analyze the ACF and PACF plots to speculate on the best stationary model for the data. They provide clear reasoning supported by observations from the plots, identifying any significant autocorrelation or partial autocorrelation patterns that suggest a particular model. Their speculation demonstrates a solid understanding of time series analysis concepts and the relationship between ACF, PACF, and model selection.             | Students attempt to speculate on the best stationary model based on the ACF and PACF plots but may struggle to provide clear reasoning or insights. Their analysis lacks coherence or depth, with limited discussion of relevant autocorrelation or partial autocorrelation patterns. They may overlook significant features in the plots or misinterpret their implications for model selection. |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (10)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3d: AR(p) fitted**                        | Students proficiently fit an AR(p) model to the yearly percentage change in US Real GDP. They correctly select an appropriate lag order p based on the ACF and PACF analysis or other model selection criteria, estimate the model parameters using appropriate methods, and interpret the results effectively.                                                                                                                                        | They may have difficulty selecting an appropriate lag order\                                                                                                                                                                                                                                                                                                                                      |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | p, estimating the model parameters accurately, or interpreting the results effectively. Their interpretation may lack depth or clarity, indicating a limited understanding of the modeling process                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (10)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3e: MA(q) fitted**                        | Students proficiently fit an MA(q) model to the yearly percentage change in US Real GDP. They correctly select an appropriate lag order p based on the ACF and PACF analysis or other model selection criteria, estimate the model parameters using appropriate methods, and interpret the results effectively.                                                                                                                                        | They may have difficulty selecting an appropriate lag order p, estimating the model parameters accurately, or interpreting the results effectively. Their interpretation may lack depth or clarity, indicating a limited understanding of the modeling process                                                                                                                                    |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (10)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3f: ARMA(p,q) fitted**                    | Students proficiently fit an ARMA(p,q) model to the yearly percentage change in US Real GDP. They appropriately select the orders p and q based on the ACF and PACF analysis or other model selection criteria, estimate the model parameters using appropriate methods, and interpret the results effectively. Their interpretation demonstrates a clear understanding of the the underlying dynamics of the data.                                    | They may have difficulty selecting an appropriate lag order p and q estimating the model parameters accurately, or interpreting the results effectively. Their interpretation may lack depth or clarity, indicating a limited understanding of the modeling process\                                                                                                                              |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                           | **Mastery (20)**                                                                                                                                                                                                                                                                                                                                                                                                                                       | **Incomplete (0)**                                                                                                                                                                                                                                                                                                                                                                                |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #### **Question 3g: Model selection**                     | Students effectively use AIC, AICc, and BIC to compare and evaluate models, presenting results in a clear table format similar to the one found in the Model Comparison section of the Time Series Notebook Ch5 Lesson 3. Their discussions on the nuance model selection evidences they understand the importance of considering the context and data generating process that is part of model specification.                                         | Students struggle to effectively use AIC, AICc, and BIC to compare and evaluate models, resulting in unclear or incomplete presentation of results or failure to address the dangers of relying solely on algorithms for model selection.\                                                                                                                                                        |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
|                                                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                        | \                                                                                                                                                                                                                                                                                                                                                                                                 |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| **Total Points**                                          | **105**                                                                                                                                                                                                                                                                                                                                                                                                                                                |                                                                                                                                                                                                                                                                                                                                                                                                   |
+-----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
