---
title: "Introduction to socialmixr"
author: "Sebastian Funk"
date: "`r Sys.Date()`"
bibliography: socialmixr.bib
nocite: '@*'
link-citations: true
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{Introduction to socialmixr}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
data.table::setDTthreads(1)
```

[socialmixr](https://github.com/epiforecasts/socialmixr) is an `R` package to derive social mixing matrices from survey data. These are particularly useful for age-structured [infectious disease models](https://en.wikipedia.org/wiki/Mathematical_modelling_of_infectious_disease). For background on age-specific mixing matrices and what data inform them, see, for example, the paper on POLYMOD by [@mossong_social_2008].

# Setup

This vignette uses the POLYMOD survey data which is included in socialmixr, and ggplot2 for plotting. To download other surveys from the [Social contact data](https://zenodo.org/communities/social_contact_data) community on Zenodo, use the [contactsurveys](https://cran.r-project.org/package=contactsurveys) package.

```{r}
library(socialmixr)
library(ggplot2)
data(polymod)
```

# The pipeline workflow

`socialmixr` provides a small set of composable functions that each perform one step in turning a survey into a contact matrix:

1. `survey[expr]` -- filter participants or contacts with an expression.
2. `assign_age_groups()` -- impute missing ages and assign participants and contacts to age groups.
3. `weigh()` -- optionally attach participant weights (day of week, age, user-defined).
4. `compute_matrix()` -- compute the mean contact matrix.
5. `symmetrise()`, `split_matrix()`, `per_capita()` -- optional post-processing.

A minimal example extracting a matrix for the UK part of POLYMOD:

```{r}
polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 1, 5, 15)) |>
  compute_matrix()
```

This produces a contact matrix with age groups 0-1, 1-5, 5-15 and 15+ years. It contains the mean number of contacts that each member of an age group (row) has reported with members of the same or another age group (column).

## Assigning age groups

`assign_age_groups()` prepares the survey for matrix computation. It imputes participant and contact ages from any available ranges, drops or keeps rows with missing ages (configurable via `missing_participant_age` and `missing_contact_age`), and adds `age.group` and `contact.age.group` columns using the `age_limits` you supply:

```{r}
uk_grouped <- polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 1, 5, 15))

head(uk_grouped$participants[, c("part_id", "part_age", "age.group")])
head(uk_grouped$contacts[, c("part_id", "cnt_age", "contact.age.group")])
```

The resulting survey object can be inspected, subset, or passed through any number of `weigh()` calls before `compute_matrix()`. If no `age_limits` are supplied, age groups are inferred from participant and contact ages.

# Surveys

Some surveys contain data from multiple countries. The POLYMOD survey, for example, contains data from:

```{r}
unique(polymod$participants$country)
```

Use the subset method `[` on a survey to restrict to one or more countries or any other column in the participant or contact data:

```{r}
polymod[country %in% c("United Kingdom", "Germany")]
```

When participants are filtered, contacts are automatically pruned to matching participants. If this subsetting is not done, the different sub-surveys contained in a dataset are combined as if they were a single sample (i.e., not applying any population-weighting by country or other correction).

# Bootstrapping

To get an idea of the uncertainty in the contact matrices, participants can be resampled with replacement. A short helper replicates participant (and matching contact) rows for each occurrence of a resampled ID, so that duplicates are preserved:

```{r}
bootstrap <- function(survey) {
  sampled_ids <- sample(
    unique(survey$participants$part_id),
    replace = TRUE
  )
  survey$participants <- survey$participants[
    list(sampled_ids), on = "part_id"
  ]
  survey$contacts <- survey$contacts[
    list(sampled_ids),
    on = "part_id",
    nomatch = NULL,
    allow.cartesian = TRUE
  ]
  survey
}

uk <- polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 1, 5, 15))

m <- suppressWarnings(
  replicate(n = 5, uk |> bootstrap() |> compute_matrix())
)
mr <- Reduce("+", lapply(m["matrix", ], function(x) x / ncol(m)))
mr
```

From these matrices, derived quantities can be obtained, for example the mean across samples as shown above.

# Demography

Obtaining symmetric contact matrices, splitting out their components (see below) and population-based participant weights require information about the underlying demographic composition of the survey population. This is represented as a `data.frame` with columns `lower.age.limit` (the lower end of each age group) and `population` (the number of people in that age group).

For recent UN World Population Prospects data, the `wpp2024` package is available from GitHub (`remotes::install_github("PPgp/wpp2024")`):

```{r eval=requireNamespace("wpp2024", quietly = TRUE), message=FALSE, warning=FALSE}
data("popAge1dt", package = "wpp2024")
uk_pop <- popAge1dt[name == "United Kingdom" & year == 2020,
  .(lower.age.limit = age, population = pop * 1000)
]
head(uk_pop)
```

Any comparable data frame will work, e.g. constructed by hand:

```{r}
custom_pop <- data.frame(
  lower.age.limit = c(0, 18, 60),
  population = c(12000000, 35000000, 20000000)
)
```

If the survey has a `country` column, `survey_country_population()` looks up country- and year-specific population data:

```{r}
survey_country_population(polymod, countries = "United Kingdom")
```

This uses the older [`wpp2017`](https://cran.r-project.org/package=wpp2017) package, an optional (Suggests) dependency that needs to be installed separately. It is kept for backwards compatibility; use more recent population data (as shown above) where possible.

# Symmetric contact matrices

Conceivably, contact matrices should be symmetric: the total number of contacts made by members of one age group with those of another should be the same as vice versa. Mathematically, if $m_{ij}$ is the mean number of contacts made by members of age group $i$ with members of age group $j$, and the total number of people in age group $i$ is $N_i$, then

$$m_{ij} N_i = m_{ji}N_j$$

Because of variation in the sample from which the contact matrix is obtained, this relationship is usually not fulfilled exactly. In order to obtain a symmetric contact matrix that fulfills it, one can use

$$m'_{ij} = \frac{1}{2N_i} (m_{ij} N_i + m_{ji} N_j)$$

To get this version of the contact matrix, pipe the matrix through `symmetrise()`, passing the population data:

```{r message=FALSE, warning=FALSE}
uk_pop <- survey_country_population(polymod, countries = "United Kingdom")

polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 1, 5, 15)) |>
  compute_matrix() |>
  symmetrise(survey_pop = uk_pop)
```

# Contact rates per capita

The contact matrix per capita $c_{ij}$ contains the social contact rates of one individual of age $i$ with one individual of age $j$, given the population details. For example, $c_{ij}$ is used in infectious disease modelling to calculate the force of infection, which is based on the likelihood that one susceptible individual of age $i$ will be in contact with one infectious individual of age $j$. The contact rates per capita are calculated as follows:

$$c_{ij} =  \tfrac{m_{ij}}{N_{j}}$$

Pipe the matrix through `per_capita()` to convert to per-capita rates. If combined with `symmetrise()`, the contact matrix $m_{ij}$ can show asymmetry if the sub-population sizes are different, but the contact matrix per capita will be fully symmetric:

$$c'_{ij} = \frac{m_{ij} N_i + m_{ji} N_j}{2N_iN_j} = c'_{ji}$$

```{r message=FALSE, warning=FALSE}
de_pop <- survey_country_population(polymod, countries = "Germany")

polymod[country == "Germany"] |>
  assign_age_groups(age_limits = c(0, 60)) |>
  compute_matrix() |>
  symmetrise(survey_pop = de_pop) |>
  per_capita(survey_pop = de_pop)
```

# Splitting contact matrices

`split_matrix()` decomposes the contact matrix into a _global_ component as well as components representing _contacts_, _assortativity_ and _demography_. The elements $m_{ij}$ of the contact matrix are modelled as

$$ m_{ij} = c q d_i a_{ij} n_j $$

where $c$ is the mean number of contacts across the whole population, $c q d_i$ is the number of contacts that a member of group $i$ makes across age groups, and $n_j$ is the proportion of the surveyed population in age group $j$. The constant $q$ is set so that $c q$ is equal to the value of the largest eigenvalue of $m_{ij}$; if used in an infectious disease model and assumed that every contact leads to infection, $c q$ can be replaced by the basic reproduction number $R_0$.

`split_matrix()` returns the assortativity matrix $a_{ij}$ in `$matrix`, with additional components `$mean.contacts` ($c$), `$normalisation` ($q$) and `$contacts` ($d_i$).

```{r}
polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 1, 5, 15)) |>
  compute_matrix() |>
  split_matrix(survey_pop = uk_pop)
```

# Filtering

The `[` method can be used to select particular participants or contacts. For example, in the `polymod` dataset, the indicators `cnt_home`, `cnt_work`, `cnt_school`, `cnt_transport`, `cnt_leisure` and `cnt_otherplace` take value 0 or 1 depending on where a contact occurred. The filter is evaluated against whichever table contains the referenced columns (participants, contacts, or both). Multiple filters can be chained:

```{r, warning=FALSE, message=FALSE}
# contact matrix for school-related contacts
polymod[cnt_school == 1] |>
  assign_age_groups(age_limits = c(0, 20, 60)) |>
  compute_matrix()

# contact matrix for work-related contacts involving physical contact
polymod[cnt_work == 1][phys_contact == 1] |>
  assign_age_groups(age_limits = c(0, 20, 60)) |>
  compute_matrix()

# contact matrix for daily contacts at home with males
polymod[cnt_home == 1][cnt_gender == "M"][duration_multi == 5] |>
  assign_age_groups(age_limits = c(0, 20, 60)) |>
  compute_matrix()
```

# Participant weights

## Temporal aspects and demography

Participant weights are commonly used to align sample and population characteristics in terms of temporal aspects and the age distribution. For example, the day of the week has been reported as a driving factor for social contact behaviour, hence to obtain a weekly average, the survey data should represent the weekly 2/5 distribution of weekend/week days. To align the survey data to this distribution, one can obtain participant weights in the form of:
$$w_{\textrm{day.of.week}} = \tfrac{5/7}{N_{\textrm{weekday}}/N} \text{  OR   } \tfrac{2/7}{N_{\textrm{weekend}}/N}$$
with sample size $N$, and $N_{weekday}$ and $N_{weekend}$ the number of participants that were surveyed during weekdays and weekend days, respectively.

Another driver of social contact patterns is age. To improve the representativeness of survey data, age-specific weights can be calculated as:
$$w_{age} = \tfrac{P_{a}\ /\ P}{N_{a}\ /\ N}$$
with $P$ the population size, $P_a$ the population fraction of age $a$, $N$ the survey sample size and $N_a$ the survey fraction of age $a$. The combination of age-specific and temporal weights for participant $i$ of age $a$ can be constructed as:
$$w_{i} = w_{\textrm{age}} * w_{\textrm{day.of.week}} $$

If the social contact analysis is based on stratification by splitting the population into non-overlapping groups, it requires the weights to be standardised so that the weighted totals within mutually exclusive cells equal the known population totals [@kolenikov_post-stratification_2016]. The post-stratification cells need to be mutually exclusive and cover the whole population. `compute_matrix()` applies this post-stratification normalisation within age groups.

`weigh()` is composable: each call multiplies new weights into the participants' `weight` column. In POLYMOD, the `dayofweek` column uses 0 for Sunday and 6 for Saturday, so weekdays are 1-5 and weekend days are 0 and 6:

```{r message=FALSE, warning=FALSE}
polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 18, 60)) |>
  weigh("dayofweek", target = c(5, 2), groups = list(1:5, c(0, 6))) |>
  weigh("part_age", target = uk_pop) |>
  compute_matrix()
```

The first `weigh()` call assigns weekday participants a total weight of 5 and weekend participants a total weight of 2 (the weekly 5/2 split). The second call post-stratifies against the population structure in `uk_pop` (passed as a data frame).

## User-defined participant weights

`weigh()` with no `target` multiplies an existing participant column directly into the weight. For instance, to give more importance to participants from large households:

```{r message=FALSE, warning=FALSE}
polymod |>
  assign_age_groups(age_limits = c(0, 18, 60)) |>
  weigh("hh_size") |>
  compute_matrix()
```

## Weight threshold

If the survey population differs extensively from the demography, some participants can end up with relatively high weights and as such, an excessive contribution to the population average. This warrants the limitation of single participant influences by a truncation of the weights. `compute_matrix()` accepts a numeric `weight_threshold` which caps the standardised weights and re-normalises so that the weight sum equals the group size. Weights close to the threshold may slightly exceed it after re-normalisation.

```{r message=FALSE, warning=FALSE}
polymod[country == "United Kingdom"] |>
  assign_age_groups(age_limits = c(0, 18, 60)) |>
  weigh("dayofweek", target = c(5, 2), groups = list(1:5, c(0, 6))) |>
  weigh("part_age", target = uk_pop) |>
  compute_matrix(weight_threshold = 3)
```


## Numerical example

With these numeric examples, we show the importance of post-stratification weights in contrast to using the crude weights directly within age-groups. We will apply the weights by age and day of week separately in these examples, though the combination is straightforward via multiplication.

### Get survey data
We start from a survey including 6 participants of 1, 2 and 3 years of age. The ages are not equally represented in the sample, though we assume they are equally present in the reference population. We will calculate the weighted average number of contacts by age and by age group, using {1,2} and {3} years of age. The following table shows the reported number of contacts per participant $i$, represented by $m_i$:

```{r, echo=FALSE}
survey_data <- data.frame(
  age = c(1, 1, 2, 2, 2, 3),
  day.of.week = as.factor(c(
    "weekend", "weekend", "weekend",
    "week", "week", "week"
  )),
  age.group = NA,
  m_i = c(3, 2, 9, 10, 8, 15)
)

# age groups 1-2 and 3
survey_data$age.group <- 1 - (survey_data$age < 3) + 1
survey_data$age.group <- as.factor(c("A", "B"))[survey_data$age.group]

knitr::kable(survey_data)
```

The summary statistics for the sample (N) and reference population (P) are as follows

```{r}
N <- 6
N_age <- c(2, 3, 1)
N_age.group <- c(5, 1)
N_day.of.week <- c(3, 3)

P <- 3000
P_age <- c(1000, 1000, 1000)
P_age.group <- c(2000, 1000)

P_day.of.week <- c(5 / 7, 2 / 7) * 3000
```

This survey data results in an unweighted average number of contacts:

```{r, echo=FALSE}
cat(paste(
  "unweighted average number of contacts:",
  round(mean(survey_data$m_i), digits = 2)
))
```

and age-specific unweighted averages on the number of contacts:
```{r, echo=FALSE}
knitr::kable(aggregate(
  m_i ~ age + age.group, data = survey_data, mean
))
```

### Weight by day of week

The following table contains the participants weights based on the survey day with and without the population and sample size constants ($w$ and $w'$, respectively). Note that the standardised weights $\tilde{w}$ and $\tilde{w'}$ are the same:

```{r, echo=FALSE}
# including population constants
survey_data$w <- NA
for (i in seq_len(nrow(survey_data))) {
  day_i <- survey_data$day.of.week[i]
  survey_data$w[i] <- (P_day.of.week[day_i] / P) / (N_day.of.week[day_i] / N)
}
survey_data$w_tilde <- survey_data$w / sum(survey_data$w) * N

# without population constants
survey_data$w_dot <- NA
for (i in seq_len(nrow(survey_data))) {
  day_i <- survey_data$day.of.week[i]
  survey_data$w_dot[i] <- (P_day.of.week[day_i]) / (N_day.of.week[day_i])
}
survey_data$w_dot_tilde <- survey_data$w_dot / sum(survey_data$w_dot) * N

# round
survey_data[, -(1:4)] <- round(survey_data[, -(1:4)], digits = 2)

# print
knitr::kable(survey_data)

# remove the 'dot' weights
survey_data$w_dot <- NULL
survey_data$w_dot_tilde <- NULL
```

Note the different scale of $w$ and $w'$, and the more straightforward interpretation of the numerical value of $w$ in terms of relative differences to apply truncation.
Using the standardised weights, we are able to calculate the weighted number of contacts:

```{r, echo=FALSE}
# add weighted number of contacts
survey_data["m_i * w_tilde"] <- survey_data$m_i * survey_data$w_tilde

# show table
knitr::kable(survey_data)

# remove the weighted number of contacts
survey_data$`m_i * w_tilde` <- NULL

cat(paste(
  "weighted average number of contacts:",
  round(
    mean(survey_data$m_i * survey_data$w_tilde),
    digits = 2
  )
))
```

If the population-based weights are directly used in age-specific groups, the contact behaviour of the 3 year-old participant, which participated during week day, is inflated due to the under-representation of week days in the survey sample. In addition, the number of contacts for 1 year-old participants is decreased because of the over-representation of weekend days in the survey. Using the population-weights within the two aggregated age groups, we obtain a more intuitive weighting for age group A, but it is still skewed for individuals in age group B. As such, this weighted average for age group B has no meaning in terms of social contact behaviour:

```{r, echo=FALSE}
knitr::kable(list(
  aggregate(m_i * w_tilde ~ age, data = survey_data, mean),
  aggregate(m_i * w_tilde ~ age.group, data = survey_data, mean)
))
```

If we subdivide the population, we need to use post-stratification weights ("w_PS") in which the weighted totals within mutually exclusive cells equal the sample size. For the age groups, this goes as follows:

```{r, echo=FALSE}
survey_data$w_PS <- NA
for (i in seq_len(nrow(survey_data))) {
  k_i <- survey_data$age.group[i]
  flag_k <- survey_data$age.group == k_i
  survey_data$w_PS[i] <- survey_data$w[i] /
    sum(survey_data$w[flag_k]) * N_age.group[k_i]
}

# round
survey_data[, -(1:4)] <- round(
  survey_data[, -(1:4)], digits = 2
)

knitr::kable(survey_data)
```

The weighted means equal:

```{r, echo=FALSE}
knitr::kable(aggregate(
  m_i * w_PS ~ age.group,
  data = survey_data, mean
))
```

### Weight by age

We repeated the example by calculating age-specific participant weights on the population and age-group level:

```{r, echo=FALSE}
survey_data$w <- NA
survey_data$w_tilde <- NA
survey_data$w_PS <- NULL
for (i in seq_len(nrow(survey_data))) {
  age_i <- survey_data$age[i]
  survey_data$w[i] <- (P_age[age_i] / P) / (N_age[age_i] / N)
}
survey_data$w_tilde <- survey_data$w / sum(survey_data$w) * N

survey_data$w_PS <- NA
for (i in seq_len(nrow(survey_data))) {
  k_i <- survey_data$age.group[i]
  flag_k <- survey_data$age.group == k_i
  survey_data$w_PS[i] <- survey_data$w[i] /
    sum(survey_data$w[flag_k]) * N_age.group[k_i]
}

# round
survey_data[, -(1:4)] <- round(
  survey_data[, -(1:4)], digits = 2
)

# print
knitr::kable(survey_data)

cat(paste(
  "weighted average number of contacts:",
  round(
    mean(survey_data$m_i * survey_data$w_tilde),
    digits = 2
  )
))
```

If the age-specific weights are directly used within the age groups, the contact behaviour for age group B is inflated to unrealistic levels and the number of contacts for age group A is artificially low:

```{r, echo=FALSE}
knitr::kable(list(
  aggregate(m_i * w_tilde ~ age, data = survey_data, mean),
  aggregate(m_i * w_tilde ~ age.group, data = survey_data, mean)
))
```

Using the post-stratification weights, we end up with:

```{r, echo=FALSE}
knitr::kable(aggregate(m_i * w_PS ~ age.group, data = survey_data, mean))
```


### Apply threshold

We start with survey data of 14 participants of 1, 2 and 3 years of age, sampled from a population in which all ages are equally present. Given the high representation of participants aged 1 year, the age-specific proportions are skewed in comparison with the reference population. If we calculate the age-specific weights and (un)weighted average number of contacts, we end up with:

```{r, echo=FALSE}
survey_data <- survey_data[c(rep(1:2, 5), 3:6), ]
survey_data$m_i[survey_data$age == 3] <- 30
rownames(survey_data) <- NULL

survey_data <- survey_data[order(survey_data$age), ]
survey_data$w <- NULL
survey_data$w_tilde <- NULL
survey_data$w_PS <- NULL
```


```{r, echo=FALSE}
N <- nrow(survey_data)
N_age <- table(survey_data$age)
N_age.group <- table(survey_data$age.group)
N_day.of.week <- table(survey_data$day.of.week)
```

```{r, echo=FALSE}
survey_data$w <- NA
survey_data$w_tilde <- NA
survey_data$w_PS <- NULL
for (i in seq_len(nrow(survey_data))) {
  age_i <- survey_data$age[i]
  survey_data$w[i] <- (P_age[age_i] / P) / (N_age[age_i] / N)
}
survey_data$w_tilde <- survey_data$w / sum(survey_data$w) * N

# round
survey_data[, -(1:4)] <- round(
  survey_data[, -(1:4)], digits = 2
)
knitr::kable(survey_data)

cat(paste(
  "unweighted average number of contacts:",
  round(mean(survey_data$m_i), digits = 2)
))
cat(paste(
  "weighted average number of contacts:",
  round(
    mean(survey_data$m_i * survey_data$w_tilde),
    digits = 2
  )
))
```

The single participant of 3 years of age has a very large influence on the weighted population average. As such, we propose to truncate the relative age-specific weights $w$ at 3. As such, the weighted population average equals:

```{r, echo=FALSE}
survey_data$w_tilde[survey_data$w_tilde > 3] <- 3
```

```{r, echo=FALSE}
survey_data$w_tilde[survey_data$w_tilde > 3] <- 3

cat(paste(
  "weighted average number of contacts after truncation:",
  round(
    mean(survey_data$m_i * survey_data$w_tilde),
    digits = 2
  )
))
```

# Plotting

## Using ggplot2

The contact matrices can be plotted by using the `geom_tile()` function of the `ggplot2` package.

```{r fig.width=4, fig.height=4}
df <- reshape2::melt(
  mr,
  varnames = c("age.group", "age.group.contact"),
  value.name = "contacts"
)
ggplot(df, aes(x = age.group, y = age.group.contact, fill = contacts)) +
  theme(legend.position = "bottom") +
  geom_tile()
```

## Using R base

The contact matrices can also be plotted with the `matrix_plot()` function as a grid of coloured rectangles with the numeric values in the cells. Heat colours are used by default, though this can be changed.

```{r fig.width=4, fig.height=4}
matrix_plot(mr)
matrix_plot(mr, color.palette = gray.colors)
```

# References