---
title: "0. Installing the safestats package"
author: "Alexander Ly, Rosanne Turner, Udo Boehm, Rianne de Heide, Peter Grunwald"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{0. Installing the safestats package}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.height = 4,
  fig.width = 8
)
```

The most recent version of the `safestats` package can be found on GitHub. Here we provide: 

1. an installation guide, and 
2. a first example on a design of experiments based on a safe anytime-valid test/e-variables. 

# 1. Installing the development version of the safestats package
Make sure that you have the most recent version of R (>= R 4.3). On Windows you also need to install RTools. Both can be found on the [cran website](https://cran.r-project.org/). A nice visual editor is [RStudio](https://posit.co/downloads). 

There are two ways to install the development version of the safestats package:

a. [By installing the GitHub version using the `remotes` packages](#installRemotes), or
b. [By installing a downloaded version of the package manually.](#installDownload) 

## a. <a name = "installRemotes">Install using the remotes package</a>
The remotes package can be installed as follows: 

```{r remotes, eval=FALSE}
install.packages("remotes")
library("remotes")
```

This then allows you to install and load the development version of `safestats` as follows: 

```{r eval=FALSE}
remotes::install_github("AlexanderLyNL/safestats", ref = "futility88")
library(safestats)
```

If this is successful then please skip the next subsection and continue with the [first example](#firstExample).

## b. <a name = "installDownload">Install using a downloaded tar.gz file</a>
Download the tar.gz file from [this dropbox link](https://www.dropbox.com/scl/fi/rh5x0todkjlcz5s7zpe5f/safestats_0.8.8.tar.gz?rlkey=rvybc4uhgclku0g3siu9vt1zz&dl=0) and save it to a folder that you can easily find. 

```{r eval=FALSE}
install.packages(path_to_file, repos = NULL, type="source")
```

# 2. <a name = "firstExample">First example: Designing a savi z-test experiment</a>

The `safestats` package workflow for safe anytime-valid inference (savi) is as follows: 

```{r, eval=FALSE}
# # PSEUDO CODE
# #   This won't work, but it just a way to explain the ideas
# designObj <- designSaviAnalysis(alternative="twoSided")
# result <- saviAnalysis(x=dat$x, designObj)
```

The design object "designObj" summarises which analysis, e.g. "designSaviZ" (z-test), "designSaviT" (t-test), or "designSaviTwoProportions" (test for two proportions), is going to be performed: 
- what type I error rate, i.e. alpha, is tolerated, 
- whether the test is directional, e.g. "twoSided", "greater", or "less",
- which type of e-variables is going to be used "eType", and
- which test defining parameter value is set.

The design function can also be provide with a minimal clinically relevant effect size, and a desired power In that case, the design object describes how many samples the experimenter should plan for. Due to optional stopping, the actual realised sample size will typically be smaller than what one should plan for, provided that there is a true effect equal or larger than the minimal clinical relevant effect size. 

<!-- Further details on optional stopping are provided in the next R Markdown document "I. Testing".  -->

The savi analysis functions, e.g. "saviZTest", "saviTTest" or saviTwoProportionsTest", scrutinises the data in view of the  design object at hand. In fact, the e-variable/savi test that are being discussed here can be performed after each observation and acted upon without over-inflating the chance of falsely rejecting the null hypothesis.

# A first example
To check whether the package is installed correctly, we run a savi two-sample z-test. 

Suppose a new educational programme was developed that claims to increase secondary school students' IQ scores by ten points. Assume further that the IQ scores are normally distributed with a population standard deviation of 12. The alternative hypothesis, that is, the claim that this educational programme causes an increase of ten IQ score points can be visualised as follows.

```{r}
xDomain <- seq(80, 150)
treatmentPopulation <- dnorm(xDomain, mean=122, sd=12)
controlPopulation <- dnorm(xDomain, mean=112, sd=12)

oldPar <- safestats::setSafeStatsPlotOptionsAndReturnOldOnes();
plot(xDomain, treatmentPopulation, type="l", lwd=2, col="red",
     ylab="Density", xlab="IQ Scores")
lines(xDomain, controlPopulation, type="l", lwd=2, col="blue")
lines(c(122, 122), c(0, 1), lty=3)
lines(c(112, 112), c(0, 1), lty=3)
```
We can visualise the difference score, which according to the alternative will be centred at ten. 

```{r}
xDomain <- seq(-20, 40)
# variance of the sum X+(-Y) is the sum of the variances
differencePopulationAlt <- dnorm(xDomain, mean=10, sd=sqrt(2)*12)
differencePopulationNull <- dnorm(xDomain, mean=0, sd=sqrt(2)*12)

oldPar <- safestats::setSafeStatsPlotOptionsAndReturnOldOnes();
plot(xDomain, differencePopulationAlt, type="l", lwd=2,
     col="black", ylab="Density", xlab="IQ Scores difference")
lines(xDomain, differencePopulationNull, lwd=2, lty=2)
lines(c(0, 0), c(0, 1), lwd=1, lty=3)
lines(c(10, 10), c(0, 1), lwd=1, lty=3)
```
The null hypothesis of no effect states that this difference is centred at zero instead, which is depicted as the broken curve. 

For the test we are going to perform we use the following design object:

```{r, eval=FALSE}
library(safestats)
sigma <- 12
designObj <- designSaviZ(meanDiffMin=10,
                         power=0.8, sigma=sigma,
                         testType="twoSample", seed=1)
designObj
```
Now look at the output of this procedure (if necessary make the console window bigger). The output show that if we would want to test the null hypothesis of no effect sequentially, then we need to plan for about n1=n2=36 participants in the treatment and control group to detect a minimal clinically relevant mean difference of 10 with 80\% power. We provide further information on the planned sample size in the "Design" R Markdown document. 

One advantage of e-variables is that you can analyse the data as they come in. Suppose that we perform the study as planned, thus, collect n1=n2=43 students as described in the note. The following code then shows the progression of e-values that quantify the  evidence against the null and the confidence interval as function of the sample size. 

```{r, eval=FALSE}
set.seed(2)
treatmentGroup <- rnorm(36, mean=122, sd=sigma)
controlGroup <- rnorm(36, mean=112, sd=sigma)

resultObj <- saviZTest(x=treatmentGroup, y=controlGroup,
                       designObj=designObj)
resultObj

plot(resultObj)
plot(resultObj, wantConfSeqPlot=TRUE, log=FALSE)
```
The resultObj shows that at n1=n2=36, we can reject the null, as the e-value is 93232, which is much larger than 1/alpha = 20, see the next R Markdown document on "Testing" for more information regarding this evidence threshold. The e-value progression suggest that we could have rejected the null hypothesis at after n1=n2=6 students already. The resultObj also shows an (anytime-valid) confidence interval (between 6.04 and 22.87) for the mean difference, which is relatively wide, but recall that the population standard deviation is 12, and it does cover the true mean difference of 10, see the R markdown document on "Anytime-valid confidence sequences" for further details.
