HARtools: Basic Use

The goal of this vignette is to describe the basic functions in the HARtools package.

HAR format

HAR stands for HTTP Archive. This is a common format for recording HTTP tracing information. This file contains a variety of information. It has a record of each object being loaded by a browser. Each of these objects’ timings is recorded. The common extension for these files is .har. The specification for this format is produced by the Web Performance Working Group of the World Wide Web Consortium (W3C).

An outline of the specification can be viewed at softwareishard.com

HARtools basics

HARtools is a lightweight package containing a number of tools to read/parse, visualise and write files in the HAR format.

Reading HAR files

readHAR is the function whoose aim is to read read/parse HAR objects. It currently can read/parse a number of formats. It takes an argument har which currently can be a character string, an R list, a URL or a file containing HAR data. It also carries out basic validation on the HAR content (checking mandatory/optional fields).

Reading/Parsing from file

The HARtools package contains a number of examples of HAR files. We will use one of these (HAR data from browsing google.com):

library(HARtools)
har <- readHAR(system.file(package = "HARtools", 
                           "exdata", "google.com.har"))
har

## --------HAR VERSION-------- 
## HAR specification version: 1.1 
## --------HAR CREATOR-------- 
## Created by: Firebug 
## version: 1.5X.0b8 
## --------HAR BROWSER-------- 
## Browser: Firefox 
## version: 3.6b6pre 
## --------HAR PAGES-------- 
## Page id: page_62143 , Page title: Google 
## --------HAR ENTRIES-------- 
## Number of entries: 5 
## REQUESTS: 
## Page: page_62143 
## Number of entries: 5 
##   -  http://www.google.cz/ 
##   -  http://www.google.cz/intl/en_com/images/logo_plain.png 
##   -  http://www.google.cz/extern_js/f/CgJjcxICY3orMAo4QUAdLCswDjgKLCswFjgULCsw... 
##   -  http://clients1.google.cz/generate_204 
##   -  http://www.google.cz/images/nav_logo7.png

names(har)

## [1] "log"

names(har[["log"]])

## [1] "version" "creator" "browser" "pages"   "entries"

So we see that a HAR object is a list with root log and children as defined per the specification.

Reading/Parsing a URL

The same HAR data we sourced from file is available online.

harData <- "http://www.janodvarko.cz/har/viewer/examples/google.com.har"
harURL <- readHAR(harData)
identical(harURL, har)

## [1] TRUE

Reading/Parsing an R character string

readHAR can read/parse an R character string

fileName <- system.file(package = "HARtools", "exdata", "google.com.har")
harString <- readChar(fileName, file.info(fileName)$size)
print(substr(harString, 1, 5*80))

## [1] "{\n  \"log\":{\n    \"version\":\"1.1\",\n    \"creator\":{\n      \"name\":\"Firebug\",\n      \"version\":\"1.5X.0b8\"\n    },\n    \"browser\":{\n      \"name\":\"Firefox\",\n      \"version\":\"3.6b6pre\"\n    },\n    \"pages\":[{\n        \"startedDateTime\":\"2010-01-02T14:51:01.186+01:00\",\n        \"id\":\"page_62143\",\n        \"title\":\"Google\",\n        \"pageTimings\":{\n          \"onContentLoad\":90,\n          \"onLoad\":245\n        }\n     "

harSTR <- readHAR(harString)
identical(harSTR, har)

## [1] TRUE

Reading/Parsing an R list

readHAR can read/parse an R list.

harL <- jsonlite::fromJSON(system.file(package = "HARtools",
                                          "exdata", "google.com.har"),
                              simplifyVector = FALSE)
harLIST <- readHAR(harL)
identical(harLIST, har)

## [1] TRUE

Writing HAR files

Parsed HAR objects of class har can be written to file using the writeHAR function:

harFile <- tempfile(fileext = ".har")
writeHAR(har, harFile)
har2 <- readHAR(harFile)
identical(har, har2)

## [1] TRUE

writeHAR exports the HAR in JSON format.

Visualising HAR data

HARviewer

HARtools includes a HARviewer function. HARviewer is a htmlwidget which utilises the PerfCascade to create a waterfall view of an HAR object.

har <- readHAR(system.file(package = "HARtools", "exdata",
                           "r-project.org.161028_W2_11MA.har"))
hv <- HARviewer(har, width="100%", height="100%")
# view in R
hv

The above HAR data was obtained from testing the r-project webpage using WebPageTest.

Embedding HAR charts in a shiny app

HARtools has a renderHARviewer function and a corresponding HARviewerOutput function which can be used in a shiny server and ui respectively to embed a HARviewer waterfall chart in a shiny app:

library(shiny)
if(interactive()){
  library(HARtools)
  har <- readHAR(system.file(package = "HARtools", "exdata",
                             "r-project.org.161028_W2_11MA.har"))
  hv <- HARviewer(har)
  
  app <- shinyApp(
    ui = fluidPage(
      HARviewerOutput("myHAR")
    ),
    server = function(input, output) {
      output$myHAR <- renderHARviewer(hv)
    }
  )  
  runApp(app)
}

Obtaining HAR data

Using a browser

Most modern browser have developer tools that often have the option to export a browsing session as HAR data see for example this article.

Using Browser Mob Proxy

Browser Mob Proxy captures performance data for web apps (via the HAR format), as well as manipulate browser behavior and traffic, such as whitelisting and blacklisting content, simulating network traffic and latency, and rewriting HTTP requests and responses.

An R client for BMP is available at https://github.com/johndharrison/bmproxy.

Using WebPageTest

WebPagetest is an open source project that is primarily being developed and supported by Google as part of their efforts to make the web faster.

WebPagetest is a tool that was originally developed by AOL for use internally and was open-sourced in 2008 under a BSD license. The platform is under active development on GitHub and is also packaged up periodically and available for download if you would like to run your own instance.

The online version at www.webpagetest.org is run for the benefit of the performance community with several companies and individuals providing the testing infrastructure around the globe.

An R client for WPT is available at https://github.com/johndharrison/webpagetestr. It allows the user to send test requests to www.webpagetest.org or any private instance of WPT a user may have access to.