--- title: "Tabulation Concepts" author: "Adrian Waddell" date: "6/12/2020" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Tabulation Concepts} \usepackage[utf8]{inputenc} %\VignetteEngine{knitr::rmarkdown} editor_options: chunk_output_type: console --- ```{r, include = FALSE} suggested_dependent_pkgs <- c("dplyr") knitr::opts_chunk$set( collapse = TRUE, comment = "#>", eval = all(vapply( suggested_dependent_pkgs, requireNamespace, logical(1), quietly = TRUE )) ) ``` ```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE) ``` ```{r, echo=FALSE} knitr::opts_chunk$set(comment = "#") ``` ```{css, echo=FALSE} .reveal .r code { white-space: pre; } ``` ## Introduction In this vignette we will introduce some theory behind using layouts for table creation. Much of the theory also holds true when using other table packages. For this vignette we will use the following packages: ```{r, message=FALSE} library(dplyr) library(tibble) library(rtables) ``` The data we use is the following, created with random number generators: ```{r} add_subgroup <- function(x) paste0(tolower(x), sample(1:3, length(x), TRUE)) set.seed(1) df <- tibble( x = rnorm(100), c1 = factor(sample(c("A", "B", "C"), 100, replace = TRUE), levels = c("A", "B", "C")), r1 = factor(sample(c("U", "V", "W"), 100, replace = TRUE), levels = c("U", "V", "W")) ) %>% mutate( c2 = add_subgroup(c1), r2 = add_subgroup(r1), y = as.numeric(2 * as.numeric(c1) - 3 * as.numeric(r1)) ) %>% select(c1, c2, r1, r2, x, y) df ``` ## Building A Table Row By Row Let's look at a table that has 3 columns and 3 rows. Each row represents a different analysis (functions `foo`, `bar`, `zoo` that return an `rcell()` object): ``` A B C ------------------------------------------------ foo_label foo(df_A) foo(df_B) foo(df_C) bar_label bar(df_A) bar(df_B) bar(df_C) zoo_label zoo(df_A) zoo(df_B) zoo(df_C) ``` The data passed to the analysis functions are a subset defined by the respective column and: ```{r} df_A <- df %>% filter(c1 == "A") df_B <- df %>% filter(c1 == "B") df_C <- df %>% filter(c1 == "C") ``` Let's do this on the concrete data with `analyze()`: ```{r} foo <- prod bar <- sum zoo <- mean lyt <- basic_table() %>% split_cols_by("c1") %>% analyze("x", function(df) foo(df$x), var_labels = "foo label", format = "xx.xx") %>% analyze("x", function(df) bar(df$x), var_labels = "bar label", format = "xx.xx") %>% analyze("x", function(df) zoo(df$x), var_labels = "zoo label", format = "xx.xx") tbl <- build_table(lyt, df) tbl ``` or if we wanted the `x` variable instead of the data frame: ``` A B C ------------------------------------------------ foo_label foo(x_A) foo(x_B) foo(x_C) bar_label bar(x_A) bar(x_B) bar(x_C) zoo_label zoo(x_A) zoo(x_B) zoo(x_C) ``` where: ```{r} x_A <- df_A$x x_B <- df_B$x x_C <- df_C$x ``` The function passed to `afun` is evaluated using argument matching. If `afun` has an argument `x` the analysis variable specified in `vars` in `analyze()` is passed to the function, and if `afun` has an argument `df` then a subset of the dataset is passed to `afun`: ```{r} lyt2 <- basic_table() %>% split_cols_by("c1") %>% analyze("x", foo, var_labels = "foo label", format = "xx.xx") %>% analyze("x", bar, var_labels = "bar label", format = "xx.xx") %>% analyze("x", zoo, var_labels = "zoo label", format = "xx.xx") tbl2 <- build_table(lyt2, df) tbl2 ``` Note that it is also possible that a function returns multiple rows with `in_rows()`: ```{r} lyt3 <- basic_table() %>% split_cols_by("c1") %>% analyze("x", function(x) { in_rows( "row 1" = rcell(mean(x), format = "xx.xx"), "row 2" = rcell(sd(x), format = "xx.xxx") ) }, var_labels = "foo label") %>% analyze("x", function(x) { in_rows( "more rows 1" = rcell(median(x), format = "xx.x"), "even more rows 1" = rcell(IQR(x), format = "xx.xx") ) }, var_labels = "bar label", format = "xx.xx") tbl3 <- build_table(lyt3, df) tbl3 ``` This is how we recommend you specify the row names explicitly. ## Tabulation With Row Structure Let's say we would like to create the following table: ``` A B C -------------------------------------- U foo(df_UA) foo(df_UB) foo(df_UC) V foo(df_VA) foo(df_VB) foo(df_VC) W foo(df_WA) foo(df_WB) foo(df_WC) ``` where `df_*` are subsets of `df` as follows: ```{r} df_UA <- df %>% filter(r1 == "U", c1 == "A") df_VA <- df %>% filter(r1 == "V", c1 == "A") df_WA <- df %>% filter(r1 == "W", c1 == "A") df_UB <- df %>% filter(r1 == "U", c1 == "B") df_VB <- df %>% filter(r1 == "V", c1 == "B") df_WB <- df %>% filter(r1 == "W", c1 == "C") df_UC <- df %>% filter(r1 == "U", c1 == "C") df_VC <- df %>% filter(r1 == "V", c1 == "C") df_WC <- df %>% filter(r1 == "W", c1 == "C") ``` further note that `df_*` are of the same class as `df`, i.e. `tibble`s. Hence `foo` aggregates the subset of our data to a cell value. Given a function `foo` (ignore the `...` for now): ```{r} foo <- function(df, labelstr = "", ...) { paste(dim(df), collapse = " x ") } ``` we can start calculating the cell values individually: ```{r} foo(df_UA) foo(df_VA) foo(df_WA) foo(df_UB) foo(df_VB) foo(df_WB) foo(df_UC) foo(df_VC) foo(df_WC) ``` Now we are still missing the table structure: ```{r} matrix( list( foo(df_UA), foo(df_VA), foo(df_WA), foo(df_UB), foo(df_VB), foo(df_WB), foo(df_UC), foo(df_VC), foo(df_WC) ), byrow = FALSE, ncol = 3 ) ``` In `rtables` this type of tabulation is done with `layouts`: ```{r} lyt4 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% analyze("x", foo) tbl4 <- build_table(lyt4, df) tbl4 ``` or if we would not want to see the `foo` label we would have to use: ```{r} lyt5 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% summarize_row_groups(cfun = foo, format = "xx") tbl5 <- build_table(lyt5, df) tbl5 ``` but now the row labels have disappeared. This is because `cfun` needs to define its row label. So let's redefine `foo`: ```{r} foo <- function(df, labelstr) { rcell(paste(dim(df), collapse = " x "), format = "xx", label = labelstr) } lyt6 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% summarize_row_groups(cfun = foo) tbl6 <- build_table(lyt6, df) tbl6 ``` ### Calculating the Mean Now let's calculate the mean of `df$y` for pattern I: ```{r} foo <- function(df, labelstr) { rcell(mean(df$y), label = labelstr, format = "xx.xx") } lyt7 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% summarize_row_groups(cfun = foo) tbl7 <- build_table(lyt7, df) tbl7 ``` Note that `foo` has the variable information hard-encoded in the function body. Let's try some alternatives returning to `analyze()`: ```{r} lyt8 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% analyze("y", afun = mean) tbl8 <- build_table(lyt8, df) tbl8 ``` Note that the subset of the `y` variable is passed as the `x` argument to `mean()`. We could also get the `data.frame` instead of the variable: ```{r} lyt9 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% analyze("y", afun = function(df) mean(df$y)) tbl9 <- build_table(lyt9, df) tbl9 ``` which is in contrast to: ```{r} lyt10 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% analyze("y", afun = function(x) mean(x)) tbl10 <- build_table(lyt10, df) tbl10 ``` where the function receives the subset of `y`. ### Group Summaries Pattern I is an interesting one as we can add more row structure (with further splits). Consider the following table: ``` A B C -------------------------------------- U u1 foo(<>) foo(<>) foo(<>) u2 foo(<>) foo(<>) foo(<>) u3 foo(<>) foo(<>) foo(<>) V v1 foo(<>) foo(<>) foo(<>) v2 foo(<>) foo(<>) foo(<>) v3 foo(<>) foo(<>) foo(<>) W w1 foo(<>) foo(<>) foo(<>) w2 foo(<>) foo(<>) foo(<>) w3 foo(<>) foo(<>) foo(<>) ``` where `<>` represents the data that is represented by the cell. So for the cell `U > u1, A` we would have the subset: ```{r} df %>% filter(r1 == "U", r2 == "u1", c1 == "A") ``` and so on. We can get this table as follows: ```{r} lyt11 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% split_rows_by("r2") %>% summarize_row_groups(cfun = function(df, labelstr) { rcell(mean(df$x), format = "xx.xx", label = paste("mean x for", labelstr)) }) tbl11 <- build_table(lyt11, df) tbl11 ``` or, if we wanted to calculate two summaries per row split: ```{r} s_mean_sd <- function(x) { in_rows("mean (sd)" = rcell(c(mean(x), sd(x)), format = "xx.xx (xx.xx)")) } s_range <- function(x) { in_rows("range" = rcell(range(x), format = "xx.xx - xx.xx")) } lyt12 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% split_rows_by("r2") %>% analyze("x", s_mean_sd, show_labels = "hidden") %>% analyze("x", s_range, show_labels = "hidden") tbl12 <- build_table(lyt12, df) tbl12 ``` Which has the following structure: ``` A B C --------------------------------------------------------- U u1 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) u2 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) u3 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) V v1 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) v2 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) v3 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) W w1 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) w2 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) w3 mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) ``` The rows `U`, `u1`, `u2`, ..., `W`, `w1`, `w2`, `w3` are label rows and the other rows (with `mean_sd` and `range`) are data rows. Currently we do not have content rows in the table. Content rows summarize the data defined by their splitting (i.e. `V > v1, B`). So if we wanted to add content rows at the `r2` split level then we would get: ``` A B C --------------------------------------------------------- U u1 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) u2 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) u3 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) V v1 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) v2 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) v3 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) W w1 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) w2 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) w3 s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) ``` where `s_cfun_2` is the content function and either returns one row via `rcell()` or multiple rows via `in_rows()`. The data represented by `<>` for the content rows is same data as for it's descendant, i.e. for the `U > u1, A` content row cell it is `df %>% filter(r1 == "U", r2 == "u1", c1 == "A")`. Note that content functions `cfun` operate only on data frames and not on vectors/variables so they must take the `df` argument. Further, a `cfun` must also have the `labelstr` argument which is the split level. This way, the `cfun` can define its own row name. In order to get the table above we can use the layout framework as follows: ```{r} s_mean_sd <- function(x) { in_rows("mean (sd)" = rcell(c(mean(x), sd(x)), format = "xx.xx (xx.xx)")) } s_range <- function(x) { in_rows("range" = rcell(range(x), format = "xx.xx - xx.xx")) } s_cfun_2 <- function(df, labelstr) { rcell(nrow(df), format = "xx", label = paste(labelstr, "(n)")) } lyt13 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% split_rows_by("r2") %>% summarize_row_groups(cfun = s_cfun_2) %>% analyze("x", s_mean_sd, show_labels = "hidden") %>% analyze("x", s_range, show_labels = "hidden") tbl13 <- build_table(lyt13, df) tbl13 ``` In the same manner, if we want content rows for the `r1` split we can do it at as follows: ```{r} lyt14 <- basic_table() %>% split_cols_by("c1") %>% split_rows_by("r1") %>% summarize_row_groups(cfun = s_cfun_2) %>% split_rows_by("r2") %>% summarize_row_groups(cfun = s_cfun_2) %>% analyze("x", s_mean_sd, show_labels = "hidden") %>% analyze("x", s_range, show_labels = "hidden") tbl14 <- build_table(lyt14, df) tbl14 ``` In pagination, content rows and label rows get repeated if a page is split in a descendant of a content row. So, for example, if we were to split the following table at `***`: ``` A B C --------------------------------------------------------- U u1 (n) s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) *** range s_range(<>) s_range(<>) s_range(<>) u2 (n) s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) ``` Then we would get the following two tables: ``` A B C --------------------------------------------------------- U u1 (n) s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) ``` and ``` A B C --------------------------------------------------------- U u1 (n) s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) range s_range(<>) s_range(<>) s_range(<>) u2 (n) s_cfun_2(<>) s_cfun_2(<>) s_cfun_2(<>) mean_sd s_mean_sd(<>) s_mean_sd(<>) s_mean_sd(<>) range s_range(<>) s_range(<>) s_range(<>) ``` ### Pattern III Let's consider the following tabulation pattern: ``` A B C ------------------------------------------------ label 1 foo(x_A) bar(x_B) zoo(x_C) label 2 foo(x_A) bar(x_B) zoo(x_C) label 3 foo(x_A) bar(x_B) zoo(x_C) ``` We will discuss that in a future release of `rtables`.