---
title: "Scraping Data From FIS Website"
author: "Abigail Smith"
format: html
---

Load needed libraries

```{r}
library(rvest)
library(chromote)
library(tidyverse)
library(lubridate)
```

Pick a page to scrape from and read it with embedded chrome browser

```{r}
# Copy and paste the pages URL into code below
url <- "https://live.fis-ski.com/cc-2257/results-pda.htm"

newpage <- ChromoteSession$new()
{
  newpage$Page$navigate(url)
  newpage$Page$loadEventFired()
}
# This opens a browser that displays the page
newpage$view()
```

After you have what you want to scarpe displayed in the browser, this code chunk will pull out all html elements. 

```{r}
elements <- newpage$Runtime$evaluate("document.querySelector('html').outerHTML")$result$value |>
  read_html()
```

```{r}
# Close the page so that your session does not time out
newpage$close()
```

Create columns for each variable using the html.elements("") function, by piping into  elements. These each use a different html key which can be identified with the Selector Gadget.

```{r}
Bib <- elements |>
  html_elements(".col_bib") |>
  html_text2()

Rank <- elements |>
  html_elements(".col_rank") |>
  html_text2()

Name <- elements |>
  html_elements(".col_name") |>
  html_text2()

NSA <- elements |>
  html_elements(".col_nsa") |>
  html_text2()

Result <- elements |>
  html_elements(".col_result") |>
  html_text2()

Diff <- elements |>
  html_elements(".col_diff") |>
  html_text2()
```

Make a data frame with all the columns that you just created. 

```{r}
x <- data.frame(Bib, Name, NSA, Rank, Result, Diff)
x <- x[x$Bib!="Bib",] # This removes any rows which are extra labels without skiers
```


Make the values in each variable numeric.

```{r}
x$Bib=as.numeric(x$Bib)
x$Rank=as.numeric(x$Rank)
x$Diff=as.numeric(x$Diff)
```

Convert the values for the Result variable from milliseconds to seconds and round them. 

```{r}
x$Result<-x$Result |> ms() |> period_to_seconds()
x$Result<-round(x$Result/60,3)
```

Right now we have data for 4 different points in the race all in one table, we want to split that up into four individual tables.


Create a vector of the distances being used as reference points for each of the four tables.

```{r}
Dist <- c(2,5.2,7.5,11)
```

Within that data frame, create a vector with the number of rows in the data frame and then for the true value of n create a vector in which those are divided by 4 to get rid of duplicates. The webpage shows results at 4 different times in the race, so this will split these up so that we have a table for each distance. 

```{r}
rows=dim(x)[1]
n=rows/4
```

Make groups for each distance and create columns for the rank and difference at each distance.

```{r}
group1 <- x[1: (n-3),]
group2 <- x[(n+1):(2*n-3),]
group3 <- x[(2*n+1):(3*n-3),]
group4 <- x[(3*n+1):(4*n-3),]

group1$Dist1 = Dist[1]
group2$Dist2 = Dist[2]
group3$Dist3 = Dist[3]
group4$Dist4 = Dist[4]

names(group1)[4:6]=c("Rank1", "Time1", "Diff1")
names(group2)[4:6]=c("Rank2", "Time2", "Diff2")
names(group3)[4:6]=c("Rank3", "Time3", "Diff3")
names(group4)[4:6]=c("Rank4", "Time4", "Diff4")
```

Join the groups into one big data frame and create a variable with the events code.

```{r}
y<-group1 |> left_join(group2) |> left_join(group3) |> left_join(group4)
y <- arrange(y, Rank4)
y$event = 2257
```

Make the data frame into a csv file so that it will be saved permanently. 

```{r}
# user specifies the name and path to the file
write.csv(y, "W10kTro_2257.csv", row.names = FALSE) 
```

