Hierarchical Heatmaps (hhmR) Vignette

Hierarchical Heatmaps (`hhm`)

The main function within the package is hhm. This function is useful if you wish to create a heatmap where the categories shown on both the x and y axis can be grouped in some way. This heatmap will order the categories by their assigned group and present both the categories and group labels along the axes. To illustrate how this function can be used, we use internal migration data from a fake country (example_migration). This country is made up of regions, with each region containing multiple counties. This hierachical structure is summarised below:

# Import dplyr for data cleaning
library(dplyr)

# Summarise hierarchical data structure
example_migration %>% group_by(`Origin Region`) %>% 
                      reframe(`Origin County` = unique(`Origin County`)) %>% 
                      setNames(c("Region","County"))
#> # A tibble: 18 × 2
#>    Region     County     
#>    <chr>      <chr>      
#>  1 Midlands   Silverlake 
#>  2 Midlands   Thornbury  
#>  3 Midlands   Maplewood  
#>  4 Midlands   Hawthorne  
#>  5 Midlands   Pinehurst  
#>  6 North      Greenridge 
#>  7 North      Windermoor 
#>  8 North      Bramblewood
#>  9 South East Stonebridge
#> 10 South East Brookfield 
#> 11 South East Ashford    
#> 12 South East Glenville  
#> 13 South East Sunnyvale  
#> 14 South East Westfield  
#> 15 South West Riverton   
#> 16 South West Meadowbrook
#> 17 South West Fairhaven  
#> 18 South West Oakdale

This dataset contains information on the number of people that have moved between these counties / regions over a given period of time. It contains five columns: an Origin County (and the Origin Region it is within), a Destination County (and the Destination Region it is within), and the number of people who have migrated between the origin and destination counties during the observation period (Migration).

# Show data
head(example_migration)
#>   Origin County Destination County Origin Region Destination Region Migration
#> 1    Greenridge         Greenridge         North              North       420
#> 2    Windermoor         Greenridge         North              North        28
#> 3   Bramblewood         Greenridge         North              North        28
#> 4    Silverlake         Greenridge      Midlands              North        37
#> 5     Thornbury         Greenridge      Midlands              North        27
#> 6     Maplewood         Greenridge      Midlands              North        35

Initial Heatmap

A useful way of visualising migration data is using a migration matrix or heatmap. However, as this data is hierarchical, it would also be useful to show how migrants have moved between regions within the same figure. This is where the hhm function can be useful. It can create a heatmap which shows the number of migrants that have moved between the different counties, with the rows and columns ordered and labelled by region.

To run the function, Origin County and Destination County are provided as lower-level categories to be shown along the y and x axes respectively (i.e. ylower and xlower). Additionally, Origin Region and Destination Region are provided as upper-level groups via which the rows and columns are ordered (i.e. yupper and xupper). Finally, Migration provides the values with which to populate the heatmap, while yttl_width and xttl_height are used to ensure there is enough space for the region names to be shown correctly.

# Create Intial heatmap
hierarchical_heatmap = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22)

# View result
hierarchical_heatmap

Remove diagonal

In the above example, the darkest colours are all shown along the diagonal. This is because, across a given period of time, the majority of the population does not migrate, meaning their origin and destination counties are the same. In order to focus on people who moved between different counties, we can remove the diagonal by setting rm_diag to TRUE.

# Remove diagonal from heatmap (i.e. hide static populations)
removed_diag         = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE)

# View result
removed_diag

Normalise legend

Sometimes it can be useful to normalise the values within a heatmap, so different datasets can be compared using the same legend. This can be easily done by setting norm_lgd to TRUE.

# Nomalise the legend
normalised_lgd       = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE,
                           norm_lgd = TRUE)

# View result
normalised_lgd

Manually define continuous colour scheme

The default colour scheme fades from blue to white. However, the legend colour scheme can be define manually by passing a vector of hexcodes to the argument cclrs. In this example, the viridis colour scheme has been provided.

# Manually define colour scheme for heatmap (uses viridis colour scheme)
viridis_12 = c("#440154FF","#482173FF","#433E85FF","#38598CFF","#2D708EFF","#25858EFF",
               "#1E9B8AFF","#2BB07FFF","#51C56AFF","#85D54AFF","#C2DF23FF","#FDE725FF")

# Assign continuous colour scheme
cont_clrs            = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE,
                           norm_lgd = TRUE,
                           cclrs = viridis_12)

# View result
cont_clrs

Break data into equal interval bins

The function also contains the option to break the data into a specified number categories, based on equal interval breaks between 0 and the maximum value within the dataset. This can be done by passing the number of desired categories to bins.

# Break legends into a specified number of bins
# (of equal intervals between 0 and the maximum value in `values`)
bins_15              = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE,
                           bins = 15)

# View result
bins_15

Manually define interval breaks

At times it might be desirable to manually define the intervals breaks between categories. For example, if the data is highly skewed or normally distributed then equal interval breaks may be inappropriate. Manual breaks can be provided by passing a vector of intervals to cbrks. In this instance, the hhmR function log_seq has been used to create a vector of logarithmically increasing values between 1 and the maximum value in the dataset (not on the diagonal).

# Manually break data into categories using user-specified intervals.
cbrks = log_seq(example_migration[example_migration[["Origin County"     ]] !=
                                  example_migration[["Destination County"]],] %>%
                .$Migration %>% max(), 12, rmv_extremes = TRUE)

# Show interval breaks
cbrks
#>  [1]  1  2  4  6  8 11 14 19 25 35

# Manually assign legend categories
legend_cats          = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE,
                           cbrks = cbrks)

# View result
legend_cats

Manually assign categoric colours

It is also possible to manually define the colour of each category by passing a vector of hexcodes to cclrs. The length of this vector must be two longer then the vector passed to cbrks.

# Manually assign colours to legend categories
cat_clrs             = hhm(df = example_migration,
                           ylower = "Origin County",
                           xlower = "Destination County",
                           yupper = "Origin Region",
                           xupper = "Destination Region",
                           values = "Migration",
                           yttl_width = 0.22,
                           xttl_height = 0.22,
                           rm_diag = TRUE,
                           cbrks = cbrks,
                           cclrs = viridis_12)

# View result
cat_clrs

Time-series heatmaps (`tshhm`)

Another common challenge with hierarchical data is visualising how it changes over time. The function tshhm is design to address this. This function is useful if you wish to create a time-series heatmap where the categories shown on the y axis can be grouped in some way. This heatmap will order the categories by their assigned group and present both the categories and group labels along the y-axis.

To illustrate this, we use immigration data between 2011-2015 for the same fake country as the previous example (example_time_series). This country is made up of the same regions, with each region containing the same counties as shown. It contains four columns: a County (and the Region it is within), the Year of observation, and the number of immigrants a county recieved in a given year (Immigration).

# Show data
head(example_time_series)
#> # A tibble: 6 × 4
#>   Region County      Year Immigration
#>   <chr>  <chr>      <dbl>       <int>
#> 1 North  Greenridge  2011        7452
#> 2 North  Greenridge  2012          NA
#> 3 North  Greenridge  2013         298
#> 4 North  Greenridge  2014        8800
#> 5 North  Greenridge  2015        4265
#> 6 North  Windermoor  2011        8016

Initial Heatmap

To run the function, the variable County is provided as the lower-level categories (lower) to be shown along the y-axis, while the variable Region is provided as the upper-level groups (upper) with which to order and group the heatmap rows. Additionally, Year is provided as the time intervals to be shown on along the axis (times), while Immigration provides the values with which to populate the heatmap (values). Finally, yttl_width and is used to ensure there is enough space for the region names to be shown correctly on the y-axis. Note that NA values are displayed as blank.

# Intial heatmap
time_series_heatmap = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            yttl_width  = 0.25)

# View result
time_series_heatmap

Sort rows in ascending order

It can sometimes be useful to arrange rows in ascending or descending order, depending on the values within the heatmap. The parameter sum_ascend allows for this functionality by arranging the rows within in group. The default option is alphabetical, which orders rows in alphabetical order from top to bottom. Other options include sum_ascend and mean_ascend, which order rows in ascending order (top to bottom) based on the row totals and row means respectively. This order can be reversed with the options sum_descend and mean_descend. Manually defining the order of upper-level groups (Region in this example) is demonstrated at the end of the vignette.

# Arrange counties within each region by total number of immigrants
# across all five years (ascending from top to bottom)
sort_ascending      = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            yttl_width  = 0.25)

# View result
sort_ascending

Normalise legend

Sometimes it can be useful to normalise the values within a heatmap, so different datasets can be compared using the same legend. This can be easily done by setting norm_lgd to TRUE.

# Nomalise the legend
normalised_lgd      = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            norm_lgd = TRUE,
                            yttl_width  = 0.25)

# View result
normalised_lgd

Manually define continuous colour scheme

The legend colour scheme can be defined manually by passing a vector of hexcodes to the argument cclrs. In this example, the viridis colour scheme has been provided.

# Assign continuous colour scheme
cont_clrs           = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            norm_lgd = TRUE,
                            cclrs = viridis_12,
                            yttl_width  = 0.25)

# View result
cont_clrs

Assign NA colours

It can sometimes be useful to visually highlight cells where data is missing. This can be done by passing a hexcode to the argument na_colour. This will cause all NA values within the dataset to be displayed in the specified colour.

# Assign colour for NA values
na_clrs             = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            norm_lgd = TRUE,
                            cclrs = viridis_12,
                            na_colour = "grey80",
                            yttl_width  = 0.25)

# View result
na_clrs

Break data into equal interval bins

# Break legends into a specified number of bins
# (of equal intervals between 0 and the maximum value in `values`)
bins_15             = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            bins = 15,
                            yttl_width  = 0.25)

# View result
bins_15

Manually define interval breaks

# Manually break data into categories using user-specified intervals.
cbrks = log_seq(example_time_series %>% .$Immigration %>% max(na.rm = TRUE),
                12, rmv_extremes = TRUE)

# Show breaks
cbrks
#>  [1]    1  113  239  382  548  743  982 1291 1725 2468

# Manually assign legend categories
legend_cats         = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            cbrks = cbrks,
                            yttl_width  = 0.25)

# View result
legend_cats

Manually assign categoric colours

It is also possible to manually define the colour of each category by passing a vector of hexcodes to cclrs. The length of this vector must be two longer then the vector passed to cbrks.

# Manually assign colours to legend categories
cat_clrs            = tshhm(df = example_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            cbrks = cbrks,
                            cclrs = viridis_12,
                            na_colour = "grey80",
                            yttl_width  = 0.25)

# View result
cat_clrs

Manually define the order of x-axis values and y-axis groups

If the variable supplied to times is numeric, it will by default sort values in ascending order from left to right along the x-axis. However, this order can be manually changed by supplying a factor to times. To demonstrate this, we have supplied the variable Year as a factor with the years 2011-2015 ordered non-chronologically.

Similarly, the groups supplied to Upper will by default be displayed alphabetically from top to bottom along the y-axis. However, this order can be manually changed by supplying a factor to upper. In the below example, we have supplied the variable Region with regions ordered from North-West to South-East.

# Manually define order of x-axis and groups using factor levels
new_time_series = example_time_series %>%
                  mutate(Year   = factor(Year,
                                         levels = c(2012,2011,2014,
                                                    2013,2015)),
                         Region = factor(Region,
                                         levels = c("North","Midlands",
                                                    "South West",
                                                    "South East")))

# Manually define order of x-axis and groups
rearrange_axes      = tshhm(df = new_time_series,
                            lower  = "County",
                            upper  = "Region",
                            times  = "Year",
                            values = "Immigration",
                            sort_lower = "sum_ascend",
                            cbrks = cbrks,
                            cclrs = viridis_12,
                            na_colour = "grey80",
                            yttl_width  = 0.25)

# View result
rearrange_axes

Hierarchical Heatmaps (hhmR) Vignette

Overview