Adventures with R - Cricket Analysis

On my journey for more interesting R packages, I stumbled onto CricketR. 
CricketR is a wonderful package found on the CRAN library HERE and written/maintained by tvganesh (GitHub link) . 

 Big Hat Tip to him for taking the Crickinfo Stats Guru website and converting the query/output to a R package

As usual, RStudio is the programming interface that I used. For ease of use, I go to Tools --> Global Options --> Appearance. I used Sky as my RStudio theme, Lucida console, font size 11 and Idle Fingers as my Editor theme. It gives a neutral black background and the fonts (comments, code, etc) is much more clearer to see

 All the files on the code are as follows,
Main Code
Player File
Venue File
Result File

 A key objective I had in this exercise
Avoid manual code runs or brute force. The package was easy enough to run one player at a time but I wanted to run it in an automated fashion for multiple players

  • The R package needs player id to run but Cricinfo provides the PlayerName. I was not able to find an easy way to get the playerid from the PlayerName. All ears if someone has an easy way to mass extract player name and player id. That would allow us to add even more players to the mix. I have around 50 right now - mostly from India but a fair bit of other players also
    • Update : After tinkering around Rvest and some other functions, I was able to figure out how to extract a player database from the cricinfo website. A different blogpost found HERE would walk everyone through the approach on that
  • I have looped the code to run for ODI, T20 and test match separately. A much more efficient approach would be to call the three R functions in the same code block and loop them through. It would not reduce the time on the code but would be more efficient

Some things I want to do on top of this code
1. I have this code running on my local PC. I would love to get someone's guidance to run this on either Azure or AWS in an automated fashion so that it runs once a week and grabs all data and outputs it out as a csv file. Would show me how the entire cloud ML layer works
2. There are functions in the R package to incorporate a forecasting module. I was not able to advance the code to that point. But I believe several ML aspects can be worked on. Some that I want to do, 

  • A forecasting approach to forecast the number of runs in the next N games at a batsman level
  • A clustering approach to cluster batsmen together

## Code Run Through

 # You need the package cricketr installed from the CRAN library. 
# install.packages("cricketr")

 library(cricketr)
library(dplyr)

library(tidyr)

 # I use a directory on my C folder as a working directory. You can use any directory
setwd('C:/Training/R/CricketAnalysis/')

 #The next pieces of code loop through the three csv files and create an unique listing of players, venue types and result types

 # Get listing of venue
VenueTable <- read.csv("VenueTable.csv")
venue_results <- list()
for (i in 1:nrow(VenueTable)) {
  
  venue_results[[i]] <- data.frame(VenueID = VenueTable$VenueID[i], stringsAsFactors = FALSE)
  
}

 Venue <- data.table::rbindlist(venue_results, fill = TRUE)
Venue <- unique(Venue[,1])

 # Players in database
PlayerTable <- read.csv("PlayerTable.csv")
player_results <- list()

 for (i in 1:nrow(PlayerTable)) {
  
  player_results[[i]] <- data.frame(playerid = PlayerTable$PlayerID[i], stringsAsFactors = FALSE)

 }

 Player <- data.table::rbindlist(player_results, fill = TRUE)
Player <- unique(Player[,1])

 # Result Type
ResultTable <- read.csv("ResultTable.csv")
result_results <- list()

 for (i in 1:nrow(ResultTable)) {
  
  result_results[[i]] <- data.frame(ResultID = ResultTable$ResultID[i], stringsAsFactors = FALSE)

}

 Result <- data.table::rbindlist(result_results, fill = TRUE)
Result <- unique(Result[,1])
rm(list= ls()[!(ls() %in% c('Result', 'Venue', 'Player', 'PlayerTable', 'VenueTable', 'ResultTable'))])

 # Data Processing for ODI
extraction_list <- list()
m <- 0

 for (i in 1:nrow(Player)) {
  
  for (j in 1:nrow(Venue)) {
    
    for (k in 1:nrow(Result)) {
      
      m <- m+1
      
      tryCatch ({
        
        post <- getPlayerDataOD(profile = Player$playerid[i], type = "batting", homeOrAway = Venue$VenueID[j], result = Result$ResultID[k]) 
        
        extraction_list[[m]] <- data.frame(Player = Player$playerid[i],
                                           Runs = post$Runs,
                                           Mins = post$Mins,
                                           BF = post$BF,
                                           Fours = post$`4s`,
                                           Sixes = post$`6s`,
                                           SR = post$SR,
                                           Pos = post$Pos,
                                           Dismissal = post$Dismissal,
                                           Inns = post$Inns,
                                           Opposition = post$Opposition,
                                           Ground = post$Ground,
                                           StartDate = post$`Start Date`,
                                           VenueType = Venue$VenueID[j], 
                                           ResultType = Result$ResultID[k], 
                                           MatchType = 'ODI', 
                                           stringsAsFactors = FALSE)
      }, error = function(e) e)
      
    }
    
  }
}
extraction1 <- data.table::rbindlist(extraction_list[!extraction_list=='NA'], fill = TRUE)
extraction1a <- left_join(extraction1, PlayerTable, by=c("Player" = "PlayerID"))
extraction1b <- left_join(extraction1a, VenueTable, by=c("VenueType" = "VenueID"))
extraction1c <- left_join(extraction1b, ResultTable, by=c("ResultType" = "ResultID"))

 extraction1 <- subset(extraction1c, select = -c(VenueType, ResultType))

 #Data clean up
## Removing rows where Player did not Bat
extraction1 <- extraction1[!(extraction1$Runs == 'DNB'),]
extraction1 <- extraction1[!(extraction1$Runs == 'TDNB'),]

 ## Convert Runs to Numeric
extraction1 <- extraction1[!(extraction1$Runs == 'absent'),]
extraction1 <- extraction1[!(extraction1$Runs == 'sub'),]
extraction1$Runs = as.numeric(gsub("\\*", "", extraction1$Runs))

 # Convert character to numerc in R
extraction1$BF <- as.numeric(as.character(extraction1$BF))

 extraction1$Fours[extraction1$Fours=="-"] <- 0
extraction1$Fours <- as.numeric(as.character(extraction1$Fours))

 extraction1$Sixes[extraction1$Sixes=="-"] <-0
extraction1$Sixes <- as.numeric(as.character(extraction1$Sixes))

 extraction1$Pos <- as.numeric(as.character(extraction1$Pos))
extraction1$Inns <- as.numeric(as.character(extraction1$Inns))

 extraction1$SR[extraction1$SR=="-"] <- 0
options(digits=3)
extraction1$SR = as.double(as.character(extraction1$SR))

 ODI <- extraction1

 # Data Processing for T20
extraction_list <- list()
m <- 0

 for (i in 1:nrow(Player)) {
  
  for (j in 1:nrow(Venue)) {
    
    for (k in 1:nrow(Result)) {
      
      m <- m+1
      
      tryCatch ({
        
        post <- getPlayerDataTT(profile = Player$playerid[i], type = "batting", homeOrAway = Venue$VenueID[j], result = Result$ResultID[k]) 
        
        extraction_list[[m]] <- data.frame(Player = Player$playerid[i],
                                           Runs = post$Runs,
                                           Mins = post$Mins,
                                           BF = post$BF,
                                           Fours = post$`4s`,
                                           Sixes = post$`6s`,
                                           SR = post$SR,
                                           Pos = post$Pos,
                                           Dismissal = post$Dismissal,
                                           Inns = post$Inns,
                                           Opposition = post$Opposition,
                                           Ground = post$Ground,
                                           StartDate = post$`Start Date`,
                                           VenueType = Venue$VenueID[j], 
                                           ResultType = Result$ResultID[k], 
                                           MatchType = 'T20', 
                                           stringsAsFactors = FALSE)
      }, error = function(e) e)
      
    }
    
  }
}

 #extraction1 <- data.table::rbindlist(extraction_list[!extraction_list=='NULL'], fill = TRUE)
extraction1 <- data.table::rbindlist(extraction_list[!extraction_list=='NA'], fill = TRUE)
extraction1a <- left_join(extraction1, PlayerTable, by=c("Player" = "PlayerID"))
extraction1b <- left_join(extraction1a, VenueTable, by=c("VenueType" = "VenueID"))
extraction1c <- left_join(extraction1b, ResultTable, by=c("ResultType" = "ResultID"))

 extraction1 <- subset(extraction1c, select = -c(VenueType, ResultType))

 #Data clean up
## Removing rows where Player did not Bat
extraction1 <- extraction1[!(extraction1$Runs == 'DNB'),]
extraction1 <- extraction1[!(extraction1$Runs == 'TDNB'),]

 ## Convert Runs to Numeric
extraction1$Runs = as.numeric(gsub("\\*", "", extraction1$Runs))

 # Convert character to numerc in R
extraction1$BF <- as.numeric(as.character(extraction1$BF))

 extraction1$Fours[extraction1$Fours=="-"] <- 0
extraction1$Fours <- as.numeric(as.character(extraction1$Fours))

 extraction1$Sixes[extraction1$Sixes=="-"] <-0
extraction1$Sixes <- as.numeric(as.character(extraction1$Sixes))

 extraction1$Pos <- as.numeric(as.character(extraction1$Pos))
extraction1$Inns <- as.numeric(as.character(extraction1$Inns))

 extraction1$SR[extraction1$SR=="-"] <- 0
options(digits=3)
extraction1$SR = as.double(as.character(extraction1$SR))

 T20 <- extraction1

 # Data Processing for Test Matches
extraction_list <- list()
m <- 0

 for (i in 1:nrow(Player)) {
  
  for (j in 1:nrow(Venue)) {
    
    for (k in 1:nrow(Result)) {
      
      m <- m+1
      
      tryCatch ({
        
        post <- getPlayerData(profile = Player$playerid[i], type = "batting", homeOrAway = Venue$VenueID[j], result = Result$ResultID[k]) 
        
        extraction_list[[m]] <- data.frame(Player = Player$playerid[i],
                                           Runs = post$Runs,
                                           Mins = post$Mins,
                                           BF = post$BF,
                                           Fours = post$`4s`,
                                           Sixes = post$`6s`,
                                           SR = post$SR,
                                           Pos = post$Pos,
                                           Dismissal = post$Dismissal,
                                           Inns = post$Inns,
                                           Opposition = post$Opposition,
                                           Ground = post$Ground,
                                           StartDate = post$`Start Date`,
                                           VenueType = Venue$VenueID[j], 
                                           ResultType = Result$ResultID[k], 
                                           MatchType = 'Test', 
                                           stringsAsFactors = FALSE)
      }, error = function(e) e)
      
    }
    
  }
}

 #extraction1 <- data.table::rbindlist(extraction_list[!extraction_list=='NULL'], fill = TRUE)
extraction1 <- data.table::rbindlist(extraction_list[!extraction_list=='NA'], fill = TRUE)
extraction1a <- left_join(extraction1, PlayerTable, by=c("Player" = "PlayerID"))
extraction1b <- left_join(extraction1a, VenueTable, by=c("VenueType" = "VenueID"))
extraction1c <- left_join(extraction1b, ResultTable, by=c("ResultType" = "ResultID"))

 extraction1 <- subset(extraction1c, select = -c(VenueType, ResultType))

 #Data clean up
## Removing rows where Player did not Bat
extraction1 <- extraction1[!(extraction1$Runs == 'DNB'),]
extraction1 <- extraction1[!(extraction1$Runs == 'TDNB'),]
extraction1 <- extraction1[!(extraction1$Runs == 'absent'),]

 ## Convert Runs to Numeric
extraction1$Runs = as.numeric(gsub("\\*", "", extraction1$Runs))

 # Convert character to numerc in R
extraction1$BF[extraction1$BF=="-"] <- 0
extraction1$BF <- as.numeric(as.character(extraction1$BF))

 extraction1$Fours[extraction1$Fours=="-"] <- 0
extraction1$Fours <- as.numeric(as.character(extraction1$Fours))

 extraction1$Sixes[extraction1$Sixes=="-"] <-0
extraction1$Sixes <- as.numeric(as.character(extraction1$Sixes))

 extraction1$Pos <- as.numeric(as.character(extraction1$Pos))
extraction1$Inns <- as.numeric(as.character(extraction1$Inns))

 extraction1$SR[extraction1$SR=="-"] <- 0
options(digits=3)
extraction1$SR = as.double(as.character(extraction1$SR))

 Test <- extraction1

 write.csv(ODI, file="ODIData.csv")
write.csv(T20, file="T20.csv")
write.csv(Test, file="Test.csv")
rm(list= ls()[!(ls() %in% c('T20','ODI','Test'))])


The final stage provides three csv files. I took the ODI file and created a Tableau Public dashboard that can be accessed HERE

Let me know your thoughts !!!

Comments

Post a Comment

Popular posts from this blog

Outsourcing - the new wave !!

First they outsourced their customer service, then their market research analysis, then teaching and then some more....... Well, if Time is to be believed, the day is not far when we have the US Army outsourcing its 'war on terror' to where else ? - INDIA !! What do these "soldiers" get - US Citizenship. According to "Fast Track" in the August 7, 2006 issue of Time. "When President Bush visited Walter Reed Army Medical Center last week to witness three U.S. soldiers seriously wounded in Iraq become U.S. citizens, he said that "if somebody's willing to risk their lives for our country, they ought to be full participants in our country." It was an executive order signed by Bush four years ago that allowed this trio, and 35,000 like them, to petition for U.S. citizenship as soon as they donned a U.S. military uniform, instead of waiting the customary three years. So far, 26,000 of them have achieved that goal. Be it patriotism or the easier p...
Read more

The Dragon vs the Tiger

Image
China may win the sprint, but India will win the marathon. I am often asked, the Dragon Vs Tiger comparisons between China and India. India is always equated with a submissive tiger (or worse, as an elephant !!!). It is true that China heads a great manufacturing empire. Foreign companies pour billions of dollars in China. Everyone from car companies to turbine manufacturers wants a heads-up in China by establishing their own manufacturing concerns there. Foreign investment, both direct and indirect, coupled with availability of plenty of natural resources and cheap labor has propelled the 'Made in China' tag to each and every part of our lives. It is extremely tough to obtain any item anywhere nowadays which does not have the 'Made in China' tag. The main question to be asked it " What if 21st century dollars are being put into a 20th century power? " Politically, China is controlled with an iron fist by communists who want to maintain the status quo at all ...
Read more

Dekh Sako to Dekh Lo, Lekin Hathoda maare bina

Blog Title Inspiration - Rediff India Review Standard Disclaimers - All the content of this post is not owned by the author of the post, it was written by his Alter Ego. Please read the external links at your own risk, Author or his alter ego not responsible for any mental trauma associated with reading too much about the Great Prof Better late than never I have been a silent spectator to the entire IIPM issue and the virtual war it has unleashed. There is no defense of IIPM and its high handed techniques. To summarize Patrix (of DesiPundit fame), the core issue here is FREEDOM. In this age where we increasingly take this simple word for granted, what IIPM is doing and has been trying to do is to take away one thing we have which is truly ours and that is FREEDOM, particularly freedom of expression , freedom of choice, the ability and the opportunity to point out flaws in an argument without losing sense of rationality. Victor Frankl once pointed out; everything can be taken fr...
Read more