Welcome!

Thanks for using package eCV! This vignette provides a high-level introduction to the functionality and usage of eCV and related functions. Please refer to Gonzalez-Reymundez et al. (2023) for more details about each implemented method.

Installation

To install, please do

install.packages("eCV")

Load package eCV

Once installed, you can load the package eCV using library.

library("eCV")
FALSE Loading required package: idr
FALSE Loading required package: mvtnorm
FALSE Loading required package: future
FALSE Loading required package: future.apply
FALSE 
FALSE 
FALSE  __________________________________________________________________
FALSE |                      ______     __                         |
FALSE |               ___   / ___\ \   / /                         |
FALSE |              / _ \ | |    \ \ / /                          |
FALSE |             |  __/ | |___  \ V /                           |
FALSE |              \___|  \____|  \_/                            |
FALSE |__________________________________________________________________|
FALSE 
FALSE         Enhanced Coefficient of Variation and IDR Extensions 
FALSE                   for Reproducibility Assessment
FALSE 
FALSE This package provides extensions and alternative methods to  IDR  to 
FALSE measure the reproducibility of omic data with an arbitrary number of 
FALSE replicates. It introduces an enhanced Coefficient of Variation (eCV)
FALSE metric to assess the likelihood of omic features being reproducible.

Usage examples

The following examples assume you have installed and loaded package tidyverse.

library("tidyverse")
FALSE ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
FALSE ✔ ggplot2 3.4.3     ✔ purrr   1.0.2
FALSE ✔ tibble  3.2.1     ✔ dplyr   1.1.2
FALSE ✔ tidyr   1.3.0     ✔ stringr 1.5.0
FALSE ✔ readr   2.1.2     ✔ forcats 0.5.1
FALSE ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
FALSE ✖ dplyr::filter() masks stats::filter()
FALSE ✖ dplyr::lag()    masks stats::lag()

Simulate data: function `simulate_data`

The simulate_data function simulates omic features, specifically designed to mimic reproducible and irreproducible groups. Building upon the copula mixture model simulations introduced in the original publication Li et al. (2011). This function produces samples of paired omic features for a specified number of replicates. The reproducibility state of each feature is determined by a binomial variable, where the mixing probabilities between two multivariate normal distributions are controlled by a vector of probabilities.

Users can tailor simulations using the scenario parameter, aligning with predefined scenarios from Li et al. (2011). These scenarios range from highly correlated and abundant reproducible features (Scenario 1) to situations with fewer, less correlated reproducible features (Scenarios 2 and 3), and even introduce a “reproducible noise” component (Scenario 4).

Users can customize the number of replicates (n_reps), the quantity of features (n_features), and the simulation scenario, while the function returns simulated data and associated parameter values in a convenient list format.

Here is an example that simulates 1,000 features across four replicates according to “Scenario 1”.

set.seed(42)
out <- simulate_data(scenario = 1, n_reps = 4, n_features = 1000)
out$sim_data %>% as.data.frame() %>% 
 mutate(`Features group` = as.character(out$sim_params$feature_group)) %>%
 ggplot(aes(x=`Rep 1`,y=`Rep 2`,color=`Features group`)) +
  geom_point(size=1, alpha=0.5) + 
  scale_color_manual(values = c( "#009CA6" , "#F4364C")) + 
  theme_classic()

Assessing feature reproducibility with different methods: function `mrep_assessment`

The mrep_assessment function is a comprehensive tool within the package eCV for evaluating the reproducibility of omic feature values across multiple sample replicates. This function acts as a wrapper for various methods, allowing users to choose between “IDR,” “gIDR,” “mIDR,” and “eCV” based on their specific needs.

When employing the traditional “IDR” method, the function utilizes the idr package’s implementation (idr::est.IDR), considering only the first two replicates regardless of the total available. For scenarios with more than two replicates, users can opt for “gIDR” and “mIDR,” both of which reduce to the traditional IDR when there are only two replicates.

When calling the “eCV” method within mrep_assessment, an “enhanced” coefficient of variation (eCV) is calculated. This metric gauges the likelihood of an omic feature being reproducible by considering the variability among replicates and the mean intensity. The eCV is calculated as \(\frac{|\sigma^2 - \mu^2|} { \mu^2}\), offering insights into the noise (\(\sigma\)) and mean (\(\mu\)) intensity characteristics of the features.

The “eCV” method further extends its analysis by making inferences based on the probabilities of eCV values originating from the group of reproducible features. It assumes a prior Normal distribution for reproducible features, sampling pseudo replicates using Probabilistic Bootstrap.

The user-friendly interface of mrep_assessment requires a numeric matrix x representing omic features across replicates, and allows customization through the method and param parameters.

The function returns a list with two elements: a numeric vector rep_index indicating reproducibility levels, and a string specifying the method used in this assessment. Additionally, users can leverage parallel computing via future and future.apply packages. This option is managed by adjusting the n_threads parameter for the “mIDR” and “eCV” methods.

Here is an example where we call function mrep_assessment interactively on the previous simulated data, using multiple methods.

# Define parameters for each method.
params <- list(
  eCV = list(max.ite = 100),
  gIDR = list(
    mu = 2,
    sigma = 1.3,
    rho = 0.8,
    p = 0.7,
    eps = 1e-3,
    max.ite = 50
  ),
  mIDR = list(
    mu = 2,
    sigma = 1.3,
    rho = 0.8,
    p = 0.7,
    eps = 1e-3,
    max.ite = 50
  )
)

# Create a list to store results
results <- NULL
# Loop through methods and calculate reproducibility
for (method in c("eCV", "gIDR", "mIDR")) {
  results <- results %>%
    bind_rows(data.frame(
      value =
        mrep_assessment(
          x = out$sim_data,
          method = method,
          param = params[[method]]
        )$rep_index,
      Method = method,
      group = out$sim_params$feature_group
    ))
}

# Plot results
 results %>% 
 mutate(group = ifelse(group == 1,"FALSE","TRUE")) %>%
 ggplot(aes(x=Method, y = value,fill=group)) + 
 scale_fill_manual(values = c( "#009CA6" , "#F4364C")) + 
 geom_boxplot() + 
 theme_classic() + 
 labs(y="Reproducibility assessment", fill="Reproducible\nfeature")

References

Gonzalez-Reymundez, Agustin, Kylie Shen, Wayne Doyle, Sichong Peng, Kasey Hutt, and Stephanie Bruns. 2023. “eCV: Enhanced Coefficient of Variation and IDR Extensions for Reproducibility Assessment of High-Throughput Experiments with Multiple Replicates.” bioRxiv. https://doi.org/10.1101/2023.12.18.572208.

Li, Qunhua, James B. Brown, Haiyan Huang, and Peter J. Bickel. 2011. “Measuring Reproducibility of High-Throughput Experiments.” The Annals of Applied Statistics 5 (3). https://doi.org/10.1214/11-AOAS466.

Introduction to the eCV package