---
title: "Data to answer a question: COVID-19 and..."
output:
  tufte::tufte_handout: default
  tufte::tufte_html: 
    tufte_features: ["fonts",  "italics"]
---

```{r setup, include=FALSE,echo=FALSE}
knitr::opts_chunk$set(echo = TRUE)
# this set-up chunk loads a series of library that might be useful. In order for it to work on your laptop, you have to 
library(ggplot2) # this is a library for plotting, following are libraries for data manipulation
library(dplyr)
library(readr)
library(tidyr)
library(gridExtra)
library(tufte) # this library provides the nice formatting
library(reticulate) # This library allows you to call Python from within R
#use_python("/opt/miniconda3/bin/python")  # This is pointing to the python on my computer, you will need to update it to yours.
use_python("/opt/anaconda3/bin/python")
```




 "Data! Data! Data! [...] I can't make bricks without clay!", famously screams Sherlock Holmes.^[Arthur Conan Doyle (1892) "The Adventure of the Copper Beeches" in *The Adventures of Sherlock Holmes*] 
 
 So, let's get some data! We are going to start with the collection of COVID-19 cases and deaths counts compiled by the New York Times. This is shared with the public in a GitHub repository^[https://github.com/nytimes/covid-19-data].  To clone a GitHub repository like the one mantained by the NYT in Rstudio is very easy (click 'file' select 'new project', choose 'version control, then 'Git' and follow instructions...). You can also only download the one file you need for the analysis below.  In order to re-run the commands below on your machine, you might need to modify them to make sure that Rstudio is looking for files in the right place (cloning the directory might take some time, so do not worry)
 
# Intro to code  chunks

R chunks are portions of R code that are included in the R markdown and executed every time the .Rmd is knitted. An R chunk starts on a new line with three backquotes  followed by `{r}` and  and ends with three backquotes  on a new line. We can choose to make the commands visible, or to view only the results, including warnings and echos, depending on the options we specify in the chunk options after `r` and in between the curly brackets.

The following code chunk loads one of the datasets available in the directory.

```{r,message=FALSE} 
us<-read_csv("us-states.csv")
```


We now take a look at the data, using a different option in the specifying the code-chunk.

```{r,echo=TRUE} 
names(us)
us[1:10,]
```

## Using Python in Rmd

You can also run Python code chunks in the same `.Rmd` document.
To do this, you need to have installed the `reticulate` package and loaded it --- which we did in the "invisible" set-up chunk. Note that the choice of using Python for this class is entirely up to you. If you do not want to install Python on your computer, please comment out (or erase) the following code chunks.


We start by loading `pandas` and reading the data.

```{python}
import pandas as pd
us_py = pd.read_csv("us-states.csv")
```

Now we use Python to compute daily new cases and find, for each state, the date with the highest number of new cases.

```{python, echo=FALSE}
us_py["date"] = pd.to_datetime(us_py["date"])
us_py = us_py.sort_values(["state", "date"])

# Daily new cases = difference from previous day within each state.
us_py["new_cases"] = us_py.groupby("state")["cases"].diff()
us_py["new_cases"] = us_py["new_cases"].fillna(0).clip(lower=0)

# For each state, keep the row where daily new cases are highest.
peak_rows = us_py.loc[us_py.groupby("state")["new_cases"].idxmax(), ["state", "date", "new_cases"]]
peak_rows = peak_rows.sort_values("new_cases", ascending=False).head(10)
peak_rows["date"] = peak_rows["date"].dt.date

print(peak_rows.to_string(index=False))
```


# Looking a graphs

Finally we  graphically display  part of the data. Note that the chunk options here include details on the size of the figure. While you can avoid specifying these, often, in order to obtain effective displays you will need to customize these values. We are using 'ggplot', which is a fairily sophisticated way of making graphics, and the commands might look a bit mysterious to you. It is always a good idea to look up the meaning of commands you might find using the Help option in Rstudio.

```{r,echo=FALSE,message = FALSE,warning=FALSE,fig.height=6,fig.width=8,fig.fullwidth = TRUE,fig.cap = "Cumulative confirmed COVID-19 cases over time by state (New York Times data)."}
p<-ggplot(us)+geom_line(aes(x=date,y=(cases)),se=FALSE,stat = "smooth") + labs(x="Date",y="Cases", title="Total COVID-19 confirmed cases over the course of the epidemic") + theme_bw() +theme(axis.text.x = element_text(angle = 90)) +facet_wrap(vars(state))
p

```
 


This is a large graph, for which we decided to use the entire width of the html file. Take a look at the code to see how we achieved that. 
The default place for a figure is in the main column. This is an OK placement for a figure that is really part of your discourse, but keep in mind that figures are not part of your text. Where ever they are placed, you need to bring them into the text by precisely referring to them. To do so you need be able to refer to their number. 


```{r,echo=FALSE,message = FALSE,warning=FALSE,fig.height=6,fig.width=8,fig.cap = "This is how I put a legend"}
p<-ggplot(us)+geom_line(aes(x=date,y=(cases)),se=FALSE,stat = "smooth") + labs(x="Date",y="Cases", title="Total COVID-19 confirmed cases over the course of the epidemic") + theme_bw() +theme(axis.text.x = element_text(angle = 90)) +facet_wrap(vars(state))
p

```

This can be tedious and error prone, fortunately, by the time we will need to do this, you will move to a version of the software that relies on the ``latex`` engine and it will take care of numbering for you.

```{r,echo=FALSE,message = FALSE,include=FALSE}
ny<-filter(us,state=="New York")
mydiff<-function(x){return(c(x[1],diff(x)))}
ny<-mutate(ny,new_cases=mydiff(cases),new_deaths=mydiff(deaths))
```
```{r,echo=FALSE,message = FALSE,warning=FALSE,fig.height=3,fig.width=4,fig.margin = TRUE,fig.cap = "This side graph focuses on the state of New York"}
p<-ggplot(ny)+ theme_bw()
p<-p+geom_line(aes(x=date,y=(new_cases)),stat="smooth",alpha=0.7)+geom_point(aes(x=date,y=(new_cases))) + labs(x="Date",y="New Cases", title="Daily new cases in New York") 
p
```


Often, we might be  interested in smaller figures. With the `tufte` package we have the option to  display them on the side of the text, by appropriately modifying the chunk options. The display on the side focuses on the state of New York, and looks at daily new cases, rather than cumulative counts. This requires us to work on the data a bit before producing a display. In the code for both graphics, there are a number of options, specifying labels, themes etc. They are an invitation for you to explore.
 



Finally, let's play around with sizes to figure out what makes for a best display of information. First of all, note that in order to have good control of your displays, you will want to have one r-chunk for each figure. 

```{r,echo=FALSE,message = FALSE,warning=FALSE,fig.height=6,fig.width=4,fig.cap = "Note the effect of some size parameters",fig.align="center"}
knitr::opts_chunk$set(out.extra='',fig.pos= "h")
p<-ggplot(ny)+ theme_bw()
p<-p+geom_line(aes(x=date,y=(new_cases)),stat="smooth",alpha=0.7)+geom_point(aes(x=date,y=(new_cases))) + labs(x="Date",y="New Cases", title="Daily new cases in New York") 
p

```
 
A final note on this document. It was created with a main goal to give you an overview of how to use R markdown, so I tried to pack it with a number of commands. In order for it to be useful, you should go back and forth between the .Rmd file and the .html one.

- Look at how to make titles of different sizes
- Make sure you note how to include links
- Find out how to write in *italic* and **bold**
- How to make a bullet points list
1. And a numbered one

