---
title: "Reproducing tables and results from Külpmann and Kuzmics (2022)"
output: rmarkdown::html_vignette
bibliography: references.bib
vignette: >
  %\VignetteIndexEntry{reproduce_paper}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE
)
```

This vignette shows how to reproduce every table, graph and test from @kulpmann2020comparing.

# Set up
The data as well as predictions for each theory mentioned in @kulpmann2020comparing are included as additional data to the package "oottest". Thus, you only need to install the package. Furthermore, we use the package xtables (@dahl2009xtable) to convert the R tables to latex tables which we use in the paper. 

```{r setup, warning=FALSE}
library("oottest")
library("xtable")
```

## Accessing the data and predictions

Data and predictions can be accessed directly after installing the oottest package. Access them as:


* data_two_action_games
* data_three_action_games
* predictions_two_action_games
* predictions_three_action_games


Please see the documentation for each dataset to learn about formatting, usage and the source.
To access the documentation for e.g., the data from the two action games, type:

```{r, eval = FALSE}
?data_two_action_games
```


# Tables and figures from the main paper
## Vuong matrix (table 1 and 2)
Table 1 and 2 consist of 6 Vuong matrices, one for each of the different types of games:

* Two-action games
    * Hawk-dove games
    * Matching pennies: asymmetric own-payoffs
    * Matching pennies: symmetric own-payoffs
* Three-action games
    * Hawk-middle-dove games
    * Rock-Scissors-Paper: asymmetric own-payoffs
    * Rock-Scissors-Paper: symmetric own-payoffs
    
The following commands creates a Vuong matrix for each of them and saves them as vuong_table_GAME_NAME. (The ranges below are to select the treatments.)
```{r}
vuong_table_hdg <- vuong_matrix(data_two_action_games[,1:10], predictions_two_action_games[,1:10,])
vuong_table_mp1 <- vuong_matrix(data_two_action_games[,11:15], predictions_two_action_games[,11:15,])
vuong_table_mp2 <- vuong_matrix(data_two_action_games[,16:20], predictions_two_action_games[,16:20,])
vuong_table_ac <- vuong_matrix(data_three_action_games[,1:10], predictions_three_action_games[,1:10,])
vuong_table_rsp1 <- vuong_matrix(data_three_action_games[,11:15], predictions_three_action_games[,11:15,])
vuong_table_rsp2 <- vuong_matrix(data_three_action_games[,16:20], predictions_three_action_games[,16:20,])
```

Now, we have six Vuong matrix, each looking similar to this:
```{r, echo = FALSE, results = 'asis'}
knitr::kable(round(vuong_table_hdg,2), caption="Hawk-Dove Games")
```


In the paper, we colored the matrices to allow for an easier overview.
To create the latex used in the paper, one only has to run the function "color_vuong_table" on each table.
It outputs latex code which can directly me used as a table in the document.

```{r, echo = FALSE, results = 'hide'}
oottest:::color_vuong_table(vuong_table_hdg)
oottest:::color_vuong_table(vuong_table_mp1)
oottest:::color_vuong_table(vuong_table_mp2)
oottest:::color_vuong_table(vuong_table_ac)
oottest:::color_vuong_table(vuong_table_rsp1)
oottest:::color_vuong_table(vuong_table_rsp2)
```


## Vuong matrix: all games (table 3)
To create the Vuong matrix using the two and three-action games together, we can not use the vuong_matrix command directly, the structure of two- and three-action games are different.

This is done here. The (hide) output is latex code of a colored Vuong matrix as above.

```{r, results='hide'}
num_theories <- dim(predictions_two_action_games)[3]
result <- matrix(, nrow = num_theories, ncol = num_theories)
for (i in 1:num_theories) {
  for (j in 1:num_theories) {
    llr_2 <- oottest:::get_llr(data_two_action_games, predictions_two_action_games[,,i], predictions_two_action_games[,,j])
    llr_3 <- oottest:::get_llr(data_three_action_games, predictions_three_action_games[,,i], predictions_three_action_games[,,j])
    variance_2 <- oottest:::get_variance_of_llr(data_two_action_games, predictions_two_action_games[,,i], predictions_two_action_games[,,j])
    variance_3 <- oottest:::get_variance_of_llr(data_three_action_games, predictions_three_action_games[,,i], predictions_three_action_games[,,j])
    result[i, j] <- (llr_2 + llr_3) / (variance_2 + variance_3)^(.5)
  }
}
colnames(result) <- colnames(predictions_two_action_games[1,,])
rownames(result) <- colnames(predictions_two_action_games[1,,])
oottest:::color_vuong_table(result)
```


### Calculating likelihoods
The third and final graph in the paper is a comparison of log-likelihood differences for four different theories.

The graph itself is created in latex using Tikz (@tantau2013a) which uses the data from R as inputs.

The data was created as follows:

Creating likelihood tables.
```{r}
llh_table_2 <- oottest:::create_likelihood_table(data_two_action_games, predictions_two_action_games)
llh_table_3 <- oottest:::create_likelihood_table(data_three_action_games, predictions_three_action_games)
llh_table_all <- cbind(llh_table_2,llh_table_3)
```

Output:
```{r, echo = FALSE, results = 'asis'}
knitr::kable(round(llh_table_2,2), caption="Loglikelihoods: two action games")
knitr::kable(round(llh_table_3,2), caption="Loglikelihoods: three action games")
```


Creating a csv file with the log-likelihood difference to the baseline (CH-RA) for each of the four theories:
NA-RA,CH-RA (normalized to 1), QLK-RA, QCH-RA
```{r}
theories <- c("NE-RA", "CH-RA", "QLK-RA", "QCH-RA")
baseline <- llh_table_all["CH-RA", ]
output <- c()
for (theory in theories) {
  lh_diff <- round(llh_table_all[theory,] - baseline, 12)
  treatment <- 1:40
  output <- cbind(treatment, lh_diff)
  write.csv(output, file= paste0(theory, ".csv"), row.names = FALSE, quote = FALSE)
}
```



## Recreating tables from the Online Appendix
To recreate the tables from the Online Appendix use:
```{r, results='hide'}
vuong_table_2 <- vuong_matrix(data_two_action_games, predictions_two_action_games)
vuong_table_3 <- vuong_matrix(data_three_action_games, predictions_three_action_games)
```

### Log-likelihoods
Creating tables from the log-likelihoods from above (requires xtable):

```{r, results='hide'}
xtable(llh_table_2, type = "latex")
xtable(llh_table_3, type = "latex")
```


### Testing theories individually
The following code gives us the results of the tests for each theory, if it generates predictions different from the observed data:
```{r, warning=FALSE, message = FALSE, results='hide'}
chi_sq_hdg <- oottest:::get_all_chi_sq(data_two_action_games[,1:10], predictions_two_action_games[,1:10,])
chi_sq_mp1 <- oottest:::get_all_chi_sq(data_two_action_games[,11:15], predictions_two_action_games[,11:15,])
chi_sq_mp2 <- oottest:::get_all_chi_sq(data_two_action_games[,16:20], predictions_two_action_games[,16:20,])
chi_sq_2 <- oottest:::get_all_chi_sq(data_two_action_games, predictions_two_action_games)
chi_sq_ac <- oottest:::get_all_chi_sq(data_three_action_games[,1:10], predictions_three_action_games[,1:10,])
chi_sq_rsp1 <- oottest:::get_all_chi_sq(data_three_action_games[,11:15], predictions_three_action_games[,11:15,])
chi_sq_rsp2 <- oottest:::get_all_chi_sq(data_three_action_games[,16:20], predictions_three_action_games[,16:20,])
chi_sq_3 <- oottest:::get_all_chi_sq(data_three_action_games, predictions_three_action_games)
```

Here the results for the hawk-dove games:
```{r, echo = FALSE, results = 'asis'}
knitr::kable(chi_sq_hdg, caption="Chi-squared test for the hawk-dove games")
```

Now, to create the latex tables as seen in the paper. These can be just copied into the tex document.
```{r, results='hide'}
table_2 <- cbind(chi_sq_hdg, chi_sq_mp1, chi_sq_mp2, chi_sq_2)
table_3 <- cbind(chi_sq_ac, chi_sq_rsp1, chi_sq_rsp2, chi_sq_3)
xtable(table_2, type = "latex", digits=c(0,2,8,2,8,2,8,2,8))
xtable(table_3, type = "latex", digits=c(0,2,8,2,8,2,8,2,8))
```

## Data and theory predictions
### Predictions
Creating tex code of a table of all predictions:
```{r, results='hide'}
pred_2 <- round(predictions_two_action_games, 3)
th_names <- names(predictions_two_action_games[1,1,])
output <- c()
for (theory in th_names) {
  output <- rbind(output, paste0("(", pred_2[1,,theory], ", ", pred_2[2,,theory], ")"))
}
colnames(output) <- names(predictions_two_action_games[1,,1])
rownames(output) <- th_names
xtable(output, type = "latex")

pred_3 <- round(predictions_three_action_games, 3)
output <- c()
for (theory in th_names) {
  output <- rbind(output, paste0("(", pred_3[1,,theory], ", ", pred_3[2,,theory], ", ", pred_3[3,,theory],  ")"))
}
colnames(output) <- names(predictions_two_action_games[1,,1])
rownames(output) <- th_names
xtable(output, type = "latex")
```

### Data
Creating tex code to display the summary of the choice data of the experiments:
```{r, results='hide'}
xtable(data_two_action_games, type = "latex", digits=0)
xtable(data_three_action_games, type = "latex", digits=0)
```


# References