Data Engineering

Working with 3D array as long-format data in R

Yang Liu / 2020-01-11

check.and.install.pkgs <- function(pkgs){ new.packages <- pkgs[!pkgs %in% installed.packages()[,"Package"]] if(length(new.packages)) install.packages(new.packages, dependencies = TRUE) suppressPackageStartupMessages(invisible(lapply(pkgs, library, character.only = TRUE))) } check.and.install.pkgs(c("data.table", "reshape2", "scatterplot3d")) Happy New…

Reproducible Workflow

Drake: powerful tool for automatic reproducible workflow

Yang Liu / 2019-09-15

drake is a powerful tool for automatic reproducible workflow. I found it a perfect match when used together with RMarkdown. There are great documentations online for drake thus here I only show a simple example working with RMarkdown. RMarkdown file could be very slow to generate if lots of calculations are involved. Any small revise makes you rerun everything. When we use drake we can do all the…

R package

Notes on writing an R package

Yang Liu / 2019-07-28

Although ‘SHAPforxgboost’ is not a package too complicated, it took me some time to get the package pass all the cran check. Now (Aug.03,2019) it is available on cran. Install by either install.packages("SHAPforxgboost") or devtools::install_github("liuyanguu/SHAPforxgboost") Use the ‘usethis’ package https://usethis.r-lib.org/ to set up the structure of the package. […]…

R package

SHAP for XGBoost in R: SHAPforxgboost

Yang Liu / 2019-07-18

I wrote the R package SHAPforxgboost to cover all the plotting functions illustrated in this post. This post serves as the vignette for the package. Please install from CRAN or Github. install.packages("SHAPforxgboost") # or devtools::install_github("liuyanguu/SHAPforxgboost") Why SHAP values SHAP’s main advantages are local explanation and consistency in global model…

Data Visualization

Shiny in Blogdown

Yang Liu / 2019-02-24

Since Blogdown is for static websites, it cannot run Shiny in rmarkdown directly. According to discussion here and document here. The only way to do it is using iframe and write outside the chunk: <iframe src="https://liuyanguu.shinyapps.io/bcl_app/" width=1000 height=800"></iframe> There is also a built-in function in knitr to do the same thing and write in the chunk. The only…

Machine Learning

Study shrinkage and DART in xgboost modeling using a simple dataset

Yang Liu / 2018-11-15

It is always a good idea to study the packaged algorithm with a simple example. Inspired by my colleague Kodi’s excellent work showing how xgboost handles missing values, I tried a simple 5x2 dataset to show how shrinkage and DART influence the growth of trees in the model. […] set.seed(123) n0 <- 5 X <- data.frame(x1 = runif(n0), x2 = runif(n0)) Y <- c(1, 5, 20, 50, 100) cbind(X,…