06-Shiny.Rmd

# User Interfaces for Health Economic Evaluation

Robert Smith^1^ & Paul Schneider^1^

\begingroup\small
*^1^ScHARR, University of Sheffield* 

*Corresponding Author: Robert Smith - rasmith3@sheffield.ac.uk*
\endgroup

This chapter follows a publication currently under open peer review at [Wellcome Open Research](https://wellcomeopenresearch.org/articles/5-69). 

To cite this publication use the following citation:
Smith R and Schneider P. Making health economic models Shiny: A tutorial. Wellcome Open Res 2020, 5:69.

More detail is available on the following open access repository: [https://github.com/RobertASmith/paper_makeHEshiny](https://github.com/RobertASmith/paper_makeHEshiny)

## Abstract

Health economic evaluation models have traditionally been built in Microsoft Excel, but more sophisticated tools are increasingly being used as model complexity and computational requirements increase. Of all the programming languages, R is most popular amongst health economists because it has a plethora of user-created packages and is highly flexible. However, even with an integrated development environments (e.g. R-Studio), R lacks a simple point-and-click user interface and therefore requires some programming ability. This might make the switch from Microsoft Excel to R seem daunting, and it might make it difficult to directly communicate results with decisions makers and other stakeholders.

The R package Shiny has the potential to resolve this limitation. It allows programmers to embed health economic models developed in R into interactive web-browser-based user interfaces. Users can specify their own assumptions about model parameters and run different scenario analyses, which, in case of regular a Markov model, can be computed within seconds. This paper provides a tutorial on how to wrap a health economic model built in R into a Shiny application. We use a 4 state Markov model developed by the Decision Analysis in R for Technologies in Health [@DARTH] group as a case-study to demonstrate main principles and basic functionality. 


A more extensive tutorial, all code, and data are provided in a GitHub repository: [https://robertasmith.github.io/healthecon_shiny/](https://robertasmith.github.io/healthecon_shiny/).


## Introduction
  
As the complexity of health economic models increase, there is growing recognition of the advantages of using high level programming languages to support statistical analysis (e.g. R, Python, C++, Julia). Depending on the model that is being used, Microsoft Excel can be relatively slow. Certain types of models (e.g. individual-level simulations) can take a very long time to run or become computationally infeasible, and some essential statistical methods can hardly be implemented at all (e.g. survival modelling, network meta analysis, value of sample information), and rely on exporting results from other programs (e.g. R, STATA, WinBUGs).

Of all the high level programming languages, R is the most popular amongst health economists [@jalal2017overview]. R is open source, supported by a large community of statisticians, data scientists and health economists. There are extensive collections of (mostly free) online resources, including packages, tutorials, courses, and guidelines. Chunks of code, model functions, and entire models are shared by numerous authors, which allow R users to quickly adopt and adapt methods and code created by others. Importantly for the UK, R is also currently the only programming environment accepted by NICE for HTA submissions, the alternative submission formats Excel, DATA/TreeAge, and WinBUGS are all software applications [@national2014guide]. 

Despite the many strengths of the script-based approach (e.g R) to decision modelling, an important limitation has been the lack of an easy-to-understand user-interface which would be useful as it "facilitates the development and communication of the model structure" [@jalal2017overview, p.743]. While it is common practice for 'spreadsheet models' to have a structured front tab, which allows decision makers to manipulate model assumptions and change parameters to assess their impact on the results, up until recently, R models had to be adapted within script files or command lines.      

Released in 2012, Shiny is an R-package which can be used to create a graphical, web browser based interface. The result looks like a website, and allows users to interact with underlying R models, without the need to manipulate the source code [@beeley2016web]. 
Shiny has already been widely adopted in many different areas and by various organisations, to present the results of statistical analysis [@gendron2016introduction]. With health economics Shiny is currently being used to conduct network meta analysis [@owen2019metainsight] and value of information analysis (@strong2014estimating; @baio2017bceaweb). 

Using Shiny, it is possible to create flexible user interfaces which allow users to specify different assumptions, change parameters, run underlying R code and visualise results. The primary benefit of this is that it makes script based computer models accessible to those with no programming knowledge - opening models up to critical inquiry from decision makers and other stakerholders [@jansen2019developing]. Other benefits come from leveraging the power of R's many publicly available packages, for example allowing for publication quality graphs and tables to be downloaded, user specific data-files uploaded and open-access data automatically updated. Shiny web applications for R health economic models seem particularly useful in cases where model parameters are highly uncertain or unknown, and where analysis may want to be conducted with hetrogeneous assumptions (e.g. for different populations). Once an R model and a Shiny application have been created, they can also be easily adapted, making it possible to quickly update the model when new information becomes available.

While, from a transparency perspective, it is preferable that models constructed in R are made open-access to improve replicability and collaboration, it is not a requirement [@hatswell2017sharing]. Sensitive and proprietary data and/or models can be shared internally, or through password-protected web applications, negating the need to email zipped folders. 

Several authors have postulated that there is considerable potential in using Shiny to support and improve health economic decision making. @incerti2019r identified web applications as being an essential part of modelling, stating that they "believe that the future of cost-effectiveness modeling lies in web apps, in which graphical interfaces are used to run script-based models" (p. 577). Similarly, @baio2017simple predicted that R Shiny web apps will be the "future of applied statistical modelling, particularly for cost-effectiveness analysis" (p.e5). Despite these optimistic prognoses, adoption of R in health economics has been slow and the use of Shiny seems to have been limited to only a few cases. A reason for this might be the lack of accessible tutorials, tailored towards an economic modeller audience.

Here, we provide a simple example of a Shiny web app, using a general 4-state Markov model. The model is based on the 'Sick-Sicker model', which has been described in detail in previous publications (@alarid2019need; @alarid2020cohort) and in open source teaching materials by the DARTH workgroup [@DARTH]. The model was slightly adapted to implement probabilistic sensitivity analysis.

## Methods

While the focus of this tutorial is on the application of Shiny for health economic models, below we provide a brief overview of the "Sick-Sicker model". For further details, readers are encouraged to consult @alarid2019need, @alarid2020cohort, @krijkamp2018microsimulation and the DARTH group website [@DARTH].

The Sick-Sicker model is a 4 state (Healthy, Sick, Sicker or Dead) Markov model. The cohort progresses through the model in cycles of equal duration, with the proportion of those in each health state in the next cycle being dependant on the proportion in each health state in the current cycle and a constant transition probability matrix.

The analysis incorporates probabilistic sensitivity analysis (PSA) by creating a data-frame of PSA inputs (one row being one set of model inputs) based on cost, utility and probability distributions using the function *f_gen_psa* and then running the model with each set of PSA inputs using the model function *f_MM_sicksicker*. We therefore begin by describing the two functions *f_gen_psa* and *f_MM_sicksicker* in more detail before moving on to demonstrate how to create a user-interface. Note that we add to the coding framework from Alarid-Escudero et al., (2019) to use the *f_* prefix for functions.

### Functions

#### Creating PSA inputs.

The f_gen_psa function returns a data-frame of probabilistic sensitivity analysis inputs: transition probabilities between health states using a beta distribution, hazard rates using a log-normal distribution, costs using a gamma distribution and utilities using a truncnormal distribution. It relies on two inputs, the number of simulations (PSA inputs), and the cost (which takes a fixed value). We set the defaults to 1000 and 50 respectively.

NOTE: in order to use the *rtruncnorm* function the user must first install and load the 'truncnorm' package using *install.packages* and *library()*. 

```{r echo = T, eval = F}
f_gen_psa <- function(n_sim = 1000, c_Trt = 50){
   
  df_psa <- data.frame(
    
    # Transition probabilities (per cycle)
    p_HS1   = rbeta(n_sim, 30, 170),        # prob Healthy -> Sick
    p_S1H   = rbeta(n_sim, 60, 60) ,        # prob Sick    -> Healthy
    p_S1S2  = rbeta(n_sim, 84, 716),        # prob Sick    -> Sicker
    
    p_HD    = rbeta(n_sim, 10, 1990)      ,  # prob Healthy -> Dead
    hr_S1   = rlnorm(n_sim, log(3),  0.01),  # rate ratio death S1 vs healthy
    hr_S2   = rlnorm(n_sim, log(10), 0.02),  # rate ratio death S2 vs healthy 
    
    # Cost vectors with length n_sim
    c_H   = rgamma(n_sim, shape = 100, scale = 20)    , # cost p/cycle in state H
    c_S1  = rgamma(n_sim, shape = 177.8, scale = 22.5), # cost p/cycle in state S1
    c_S2  = rgamma(n_sim, shape = 225, scale = 66.7)  , # cost p/cycle in state S2
    c_D   = 0                                         , # cost p/cycle in state D
    c_Trt = c_Trt,                                   # cost p/cycle of treatment
    
    # Utility vectors with length n_sim 
    u_H   = rtruncnorm(n_sim, mean =    1, sd = 0.01, b = 1), # utility when healthy
    u_S1  = rtruncnorm(n_sim, mean = 0.75, sd = 0.02, b = 1), # utility when sick
    u_S2  = rtruncnorm(n_sim, mean = 0.50, sd = 0.03, b = 1), # utility when sicker
    u_D   = 0                                               , # utility when dead
    u_Trt = rtruncnorm(n_sim, mean = 0.95, sd = 0.02, b = 1)  # utility when being treated
  )
  
  return(df_psa)
}

```

#### Running the model for a specific set of PSA inputs
The function *f_MM_sicksicker* makes use of the *with* function which applies an expression (in this case the rest of the code) to a dataset (in this case params, which will be a row of PSA-inputs). It uses the params (one row of PSA inputs) to create a transition probability matrix *m_P*, and then moves the cohort through the simulation one cycle at a time, recording the proportions in each health state in a markov trace *m_TR* and applying the transition matrix to calculate the proportions in each health state in the next period *m_TR[t+1,]*. The function returns a vector of five results: Cost with no treatment, Cost with treatment, QALYs with no treatment and QALYs with treatment and an ICER. In this simple example treatment only influences utilities and costs, not transition probabilities

```{r echo = T, eval = F}

f_MM_sicksicker <- function(params) {
  with(as.list(params), {
    
    # compute internal parameters as a function of external parameter
    r_HD    = - log(1 - p_HD) # rate of death in healthy
    r_S1D   = hr_S1 * r_HD 	  # rate of death in sick
    r_S2D   = hr_S2 * r_HD  	# rate of death in sicker
    p_S1D   = 1 - exp(-r_S1D) # probability to die in sick
    p_S2D   = 1 - exp(-r_S2D) # probability to die in sicker
    
    # calculate discount weight for each cycle based on discount rate d_r
    v_dwe <- v_dwc <- 1 / (1 + d_r) ^ (0:n_t) 
    
    # create transition probability matrix for NO treatment
    m_P <- matrix(0,
                  nrow = n_states, ncol = n_states,
                  dimnames = list(v_n, v_n))
    # fill in the transition probability array
    ### From Healthy
    m_P["H", "H"]  <- 1 - (p_HS1 + p_HD)
    m_P["H", "S1"] <- p_HS1
    m_P["H", "D"]  <- p_HD
    ### From Sick
    m_P["S1", "H"]  <- p_S1H
    m_P["S1", "S1"] <- 1 - (p_S1H + p_S1S2 + p_S1D)
    m_P["S1", "S2"] <- p_S1S2
    m_P["S1", "D"]  <- p_S1D
    ### From Sicker
    m_P["S2", "S2"] <- 1 - p_S2D
    m_P["S2", "D"]  <- p_S2D
    ### From Dead
    m_P["D", "D"] <- 1
    
    # create Markov trace (n_t + 1 because R doesn't understand  Cycle 0)
    m_TR <- matrix(NA, nrow = n_t + 1 , ncol = n_states, 
                   dimnames = list(0:n_t, v_n))     
    
    m_TR[1, ] <- c(1, 0, 0, 0)          # initialize Markov trace
    
    ############# PROCESS ###########################################
      
    
    for (t in 1:n_t){ # throughout the number of cycles
      # estimate the Markov trace for cycle the next cycle (t + 1)
      m_TR[t + 1, ] <- m_TR[t, ] %*% m_P           
    }
    
    ############ OUTPUT  ###########################################
    
    # create vectors of utility and costs for each state
    v_u_trt    <- c(u_H, u_Trt, u_S2, u_D)
    v_u_no_trt <- c(u_H, u_S1, u_S2, u_D)
    
    v_c_trt    <- c(c_H, c_S1 + c_Trt, c_S2 + c_Trt, c_D)
    v_c_no_trt <- c(c_H, c_S1, c_S2, c_D)
    
    # estimate mean QALys and costs
    v_E_no_trt <- m_TR %*% v_u_no_trt
    v_E_trt    <- m_TR %*% v_u_trt
    
    v_C_no_trt <- m_TR %*% v_c_no_trt
    v_C_trt    <- m_TR %*% v_c_trt
    
    ### discount costs and QALYs
    te_no_trt <- t(v_E_no_trt) %*% v_dwe  # 1x31 %*% 31x1 -> 1x1
    te_trt    <- t(v_E_trt) %*% v_dwe
    
    tc_no_trt <- t(v_C_no_trt) %*% v_dwc
    tc_trt    <- t(v_C_trt)    %*% v_dwc
    
    results <- c("Cost_NoTrt" = tc_no_trt, 
                 "Cost_Trt"   = tc_trt, 
                 "QALY_NoTrt" = te_no_trt, 
                 "QALY_Trt"   = te_trt,
                 "ICER"       = (tc_trt - tc_no_trt)/(te_trt - te_no_trt))
    
    return(results)
  }
  )
}
```

### Creating the model wrapper

When using a web application it is likely that the user will want to be able to change parameter inputs and re-run the model. In order to make this simple, we recommend wrapping the entire model into a function. We call this function *f_wrapper*, using the prefix *f_* to denote that this is a function.

The wrapper function has as its inputs all the parameters which we may wish to vary using R-Shiny. We set the default values to those of the base model in any report/publication. The model then generates PSA inputs using the *f_gen_psa* function, creates an empty table of results, and runs the model for each set of PSA inputs (a row from *df_psa*) in turn. The function then returns the results in the form of a dataframe with n=5 columns and n=psa rows. The columns contain the costs and qalys for treatment and no treatment for each PSA run, as well as an ICER for that PSA run.

```{r, echo = T, eval = F}
f_wrapper <- function(
  
  #================================================================
  #                       User adjustable inputs
  #================================================================
  n_age_init = 25,   # age at baseline default is 25
  n_age_max  = 110,  # maximum age of follow up default is 110
  d_r     = 0.035,   # discount rate for costs & QALYS (NICE 3.5%)
  n_sim   = 1000,    # number of simulations default 1000
  c_Trt   = 50       # cost of treatment default 50

  ){
  
  #================================================================
  #                       Unadjustable inputs
  #================================================================
  n_t <- n_age_max - n_age_init # time horizon, number of cycles
  v_n <- c("H", "S1", "S2", "D") # the 4 health states of the model:
  n_states <- length(v_n) # number of health states 

  #================================================================
  #                       Create PSA Inputs
  #================================================================
  
  df_psa <- f_gen_psa(n_sim = n_sim, c_Trt =  c_Trt)

  #================================================================
  #                       Run PSA
  #================================================================

  # Initialize matrix of results outcomes
  m_out <- matrix(NaN, 
                 nrow = n_sim, 
                 ncol = 5,
                 dimnames = list(1:n_sim,c("Cost_NoTrt", "Cost_Trt",
                                           "QALY_NoTrt", "QALY_Trt",
                                           "ICER")))
  # loop through psa inputs running the model for each.
  for(i in 1:n_sim){
    # store results in one row of results matrix
    m_out[i,] <- f_MM_sicksicker(df_psa[i, ])
    
    # display the progress of the simulation
    cat('\r', paste(round(i/n_sim * 100), "% done", sep = " "))       
  }
  
  #================================================================
  #                      Return results
  #================================================================
  
  df_out <- as.data.frame(m_out) # convert matrix to dataframe (for plots)
  
  return(df_out) # output the dataframe from the function
  
  }

```

### Integrating into R-Shiny

The method so far has involved wrapping the model into a function, which takes some inputs and returns a single data-frame output. The next step is to integrate the model function into a Shiny web-app. This is done within a single R file, which we call *app.R*. This can be found here: [https://github.com/RobertASmith/healthecon_shiny/tree/master/App](https://github.com/RobertASmith/healthecon_shiny/tree/master/App).

The app.R script has three main parts, each are addressed in turn below:
- set-up (getting everything ready so the user-interface and server can be created)
- user interface (what people will see)
- server (stuff going on in the background)

As shown in Figure 1 below, the parameters which are varied in the user interface are a subgroup of inputs into the wrapped up model (the rest of the inputs coming from within the server). The model function has one output, a table of results, which are used within the server to create tables and plots. These are sent back to the user interface to be displayed as shiny outputs. 

![Diagram depecting how the code for the Sick-Sicker app is structured](_bookdown_files/bookdown-demo_files/figures-shiny/shiny_function_app.PNG)

#### Set-up

The set-up is relatively simple, load the R-Shiny package from your library so that you can use the *shinyApp* function. The next step is to use the *source* function in baseR to run the script which creates the *f_wrapper* function, being careful to ensure your relative path is correct ('./wrapper.R' should work if the app.R file is within the same folder). The function *shinyApp* at the end of the app file is reliant on the *shiny* package so please ensure that the *shiny* package is installed, using *install.packages("shiny")* if it is not. 

```{r, echo = T, eval = F}
# install.packages("shiny") # necessary if you don't already have the function 'shiny' installed.

# we need the function shiny installed, this loads it from the library.
library(shiny)             

# source the wrapper function.
source("./wrapper.R")   

```

#### User Interface

The user interface is extremely flexible, we show the code for a very simple structure (fluidpage) with a sidebar containing inputs and a main panel containing outputs. We have done very little formatting in order to minimize the quantity of code while maintaining basic functionality. In order to get an aesthetically pleasing application we recommend much more sophisticated formatting, relying on CSS, HTML and Javascript.

The example user interface displayed in Figure 2 and online at [https://robertasmith.shinyapps.io/sick_sicker/](https://robertasmith.shinyapps.io/sick_sicker/). It is made up of two components, a titlepanel and a sidebar layout display. The sidebarLayout display has within it a sidebar and a main panel. These two components are contained within the *fluidpage* function which creates the user interface (ui).

![ScreenPrint of Sick-Sicker user interface](_bookdown_files/bookdown-demo_files/figures-shiny/sick_sicker_app.PNG)


The title panel contains the title "Sick Sicker Model in Shiny", the sidebar panel contains two numeric inputs and a slider input ("Treatment Cost", "PSA runs", "Initial Age") and an Action Button ("Run / update model"). 

The values of the inputs have ID tags (names) which are recognised and used by the server function, we denote these with the prefix "SI" to indicate they are 'Shiny Input' objects (*SI_c_Trt*, *SI_n_sim*, *SI_n_age_init*). Note that this is an addition of the coding framework provided by Alarid-Escudero et al., (2019).

The action button also has an id, this is not an input into the model wrapper (f_wrapper) so we leave out the SI and call it "run_model".

The main panel contains two objects which have been output from the server: *tableOutput("SO_icer_table")* is a table of results, and *plotOutput("SO_CE_plane")* is a cost-effectiveness plane plot. It is important that the format (e.g. tableOutput) matches the format of the object from the server (e.g. *SO_icer_table*). Again, the *SO* prefix reflects the fact that these are Shiny Outputs. The two h3() functions are simply headings which appear as "Results Table" and "Cost-effectiveness Plane".



```{r, echo = T, eval = F}
  #================================================================
  #                   Create User Interface
  #================================================================

ui <- fluidPage(    # creates empty page, which we will fill
  
  titlePanel("Sick Sicker Model in Shiny"),   # title of app
  
  # SIDEBAR
  sidebarLayout(    # indicates layout is going to be a sidebar-layout
    
    sidebarPanel( # open sidebar panel
    
    numericInput(inputId = "SI_c_Trt",      # id of input, used in server
                 label = "Treatment Cost",  # label next to numeric input
                 value = 200,               # initial value
                 min = 0,                   # minimum value allowed
                 max = 400),                # maximum value allowed
    
    numericInput(inputId = "SI_n_sim",      # id of input, used in server
                 label = "PSA runs",        # label next to numeric input
                 value = 1000,              # initial value
                 min = 0,                   # minimum value allowed
                 max = 400),                # maximum value allowed
    
    sliderInput(inputId = "SI_n_age_init",  # id of input, used in server
                label = "Initial Age",      # label next to numeric input
                value = 25,                 # initial value
                min = 10,                   # minimum value allowed
                max = 80),                  # maximum value allowed
    

    actionButton(inputId = "run_model",     # id of action button, used in server
                 label   = "Run model")     # action button label (on button)
                       
                ),  # close sidebarPanel
    
  mainPanel(                                # open main panel
    
    h3("Results Table"),                    # heading (results table)                
    
    tableOutput(outputId = "SO_icer_table"),   # tableOutput id = icer_table, from server
    
    h3("Cost-effectiveness Plane"),         # heading (Cost effectiveness plane)
    
    plotOutput(outputId = "SO_CE_plane")       # plotOutput id = CE_plane, from server
    
            ) # close mainpanel    
      
      ) # close sidebarlayout
                     
  ) # close UI fluidpage

```

#### Server

The server is marginally more complicated than the user interface. It is created by a function with inputs and outputs. The observe event indicates that when the action button (run_model) is pressed the code within the curly brackets is run. The code will be re-run if the button is pressed again. 

The first thing that happens when the run_model button is pressed is that the model wrapper function *f_wrapper* is run with the user interface inputs (*SI_c_Trt*, *SI_n_age_init*, *SI_n_sim*) as inputs to the function. The *input$* prefix indicates that the objects have come from the user interface. The results of the model are stored as the dataframe object *df_model_res*.

The ICER table is then created and output (note the prefix *output$*) in the object *SO_icer_table*. See previous section on the user interface and note that the *tableOutput* function has as an input *SO_icer_table*. The function *renderTable* rerenders the table continuously so that the table always reflects the values from the data-frame of results created above. In this simple example we have created a table of results using code within the script. In reality we would generally use a custom function which creates a publication quality table which is aesthetically pleasing. There are numerous packages which provide this functionality (e.g. BCEA, Darthpack, Heemod)

The cost-effectiveness plane is created in a similar process, using the *renderPlot* function to continuously update a plot which is created using baseR plot function using incremental costs and QALYs calculated from the results dataframe *df_model_res*. For aesthetic purposes we recommend this is replaced by a ggplot or plotly plot which has much improved functionality. Again, there are packages available for this (e.g. Heemod, BCEA, Darthpack)

```{r, echo = T, eval = F}
  #================================================================
  #                     Create Server Function
  #================================================================

server <- function(input, output){   # server = function with two inputs
  
 observeEvent(input$run_model,       # when action button pressed ...
              ignoreNULL = F, {
    
  # Run model wrapper function with the Shiny inputs and store as data-frame 
  df_model_res = f_wrapper(c_Trt = input$SI_c_Trt,
                           n_age_init = input$SI_n_age_init,
                           n_sim = input$SI_n_sim)
    
  
    #--- CREATE COST EFFECTIVENESS TABLE ---#
    output$SO_icer_table <- renderTable({ # this continuously updates table
      
      df_res_table <- data.frame( # create dataframe
        
        Option =  c("Treatment","No Treatment"), 
        
        QALYs  =  c(mean(df_model_res$QALY_Trt),mean(df_model_res$QALY_NoTrt)),
        
        Costs  =  c(mean(df_model_res$Cost_Trt),mean(df_model_res$Cost_NoTrt)),
        
        Inc.QALYs = c(mean(df_model_res$QALY_Trt) - mean(df_model_res$QALY_NoTrt),NA),
        
        Inc.Costs = c(mean(df_model_res$Cost_Trt) - mean(df_model_res$Cost_NoTrt),NA),
        
        ICER = c(mean(df_model_res$ICER),NA)
      )
      
      # round the dataframe to two digits so looks tidier
      df_res_table[,2:6] <- round(df_res_table[,2:6],digits = 2) 
      
      # print the results table
      df_res_table

      }) # table plot end.
    
    
   #---  CREATE COST EFFECTIVENESS PLANE ---#
    output$SO_CE_plane <- renderPlot({ # render plot repeatedly updates.
      
      # calculate incremental costs and qalys from results dataframe
      df_model_res$inc_C <- df_model_res$Cost_Trt - df_model_res$Cost_NoTrt
      df_model_res$inc_Q <- df_model_res$QALY_Trt - df_model_res$QALY_NoTrt
      
      # create cost effectiveness plane plot
      plot(x = df_model_res$inc_Q, # x axis incremental QALYS
           y = df_model_res$inc_C, # y axis incremental Costs
           #label axes
           xlab = "Incremental QALYs", 
           ylab = "Incremental Costs", 
           
           # set xlimits and ylimits for plot.
           xlim = c(min(df_model_res$inc_Q,df_model_res$inc_Q*-1),
                    max(df_model_res$inc_Q,df_model_res$inc_Q*-1)),
           ylim = c(min(df_model_res$inc_C,df_model_res$inc_C*-1),
                    max(df_model_res$inc_C,df_model_res$inc_C*-1)),
           # include y and y axis lines.
           abline(h = 0,v=0)
          ) # plot end 
    }) # renderplot end
    
 }) # Observe Event End
    
    
  } # Server end
  
```  

#### Running the app
The app can be run within the R file using the function *shinyApp* which depends on the *ui* and *server* which have been created and described above. Running this creates a Shiny application in the local environment (e.g. your desktop). In order to deploy the application onto the web the app needs to be *published* using the publish button in the top right corner of the R-file in RStudio (next to run-app). A step by step guide to this process can be found on the R-Shiny website [https://shiny.rstudio.com/deploy/](https://shiny.rstudio.com/deploy/). 

```{r, echo = T, eval = F}
## ----- run app------

shinyApp(ui, server)
```

#### Additional Functionality

The example Sick-Sicker web-app which has been created is a simple, but functional, R-Shiny user interface for a health economic model. There are a number of additional functionalities which we have used for various projects:

- ability to fully customise the aesthetics of the user interface similar to standard web-design.

- ability to upload files containing input parameters to the app before running the model.

- ability to download figures and tables from the app.

- ability to download a full markdown report which updates all numbers, tables and figures in the report based on the user inputs and model outputs.

- ability to select the comparitor and treatment(s) where multiple treatments exist.

It is also possible to integrate network meta-analysis (NMA), the economic model, and value of information analysis (VOI) in a single application. After selecting NMA inputs (e.g. a subgroup of effectiveness studies) and economic model inputs (e.g. costs of treatments) and then clicking the 'run' button, a user would be presented with results of the NMA, economic model and VOI in one simple user-interface. They would then be able to download a report with all of the relevant information updated (e.g. numbers, tables, figures). We believe this is the future of health economics.


## Discussion

In this paper, we demonstrated how to generate a user-friendly interface for an economic model programed in R, using the Shiny package. This tutorial shows that the process is relatively simple and requires limited additional programming knowledge than that required to build a decision model in R.

The movement towards script based health economic models with web based user interfaces is particularly useful in situations where a general model structure has been created with a variety of stakeholders in mind, each of which may have different input parameters and wish to conduct sensitivity analysis specific to their decision. For example the World Health Organisation Department of Sexual and Reproductive Health and Research recently embedded a Shiny application into their website  [https://srhr.org/fgmcost/cost-calculator/](https://srhr.org/fgmcost/cost-calculator/). The application runs a heemod model [@heemod] in R in an external server, and allows users to select their country and change country specific input parameters, run the model and display results. The process of engagement, the ability to 'play' with the model and test the extremes of the decision makers' assumptions gives stakeholders more control over models, making them feel less like black boxes. While there is a danger that a mis-informed stakeholder may make a mistake in their choice of parameter, we should remember that the role of the model is to inform decision-makers not instruct them ... and besides: it is simple to limit the range that parameter inputs can take.

The authors' experience of creating user-interfaces for decision models has led us to the conclusion that the most efficient method is to work iteratively, first ensuring that the model is working as intended before making incremental changes to the user interface and server one item at a time. While experienced programmers can make substantial time savings by combining multiple steps we have found that the time taken to correct mistakes far outweighs the time savings associated with combining steps.

There are several challenges that exist with the movement toward script based models with web-based user-interfaces. The first is the challenge of upskilling health economists used to working in Microsoft Excel. We hope that this tutorial provides a useful addition to previous tutorials demonstrating how to construct decsion models in R [@alarid2020cohort]. A second, and crucial challenge to overcome, is a concern about deploying highly sensitive data and methods to an external server. While server providers such as ShinyIO provide assurances of SSR encryption and user authentication clients with particularly sensitive data may still have concerns. This problem can be avoided in two ways: firstly if clients have their own server and the ability to deploy applications they can maintain control of all data and code, and secondly the application could simply not be deployed, and instead simply created during a meeting using code and data shared in a zip file. Finally, a challenge (and opportunity) exists to create user-interfaces that are most user-friendly for decision makers in this field, this is an area of important research which can be used to inform teaching content for years to come.

## Conclusion

The creation of web application user interfaces for health economic models constructed in high level programming languages should improve their usability, allowing stakeholders and third parties with no programming knowledge to conduct their own sensitivity analysis remotely. This tutorial provides a reference for those attempting to create a user interface for a health economic model created in R. Further work is necessary to better understand how to design interfaces which best meet the needs of different decision makers. 

## Creating a downloadable report from Shiny (Serdar Korur)

After making an analysis on a Shiny app users may want to create a report of their results with all the parameter values they used when creating their Health Economic model.

To add this functionality to a Shiny app we need to add **a download button** to the Shiny UI and **a downloadhandler()** at the server side. downloadhandler() on the server side will execute knitting on demand a separate rmd document. 

We will also need to create a separate rmd file, we may call it report.Rmd. In this file, we can include the relevant parameters which we want to be written on the report. For example, the model specific parameters users specified after testing their own assumptions and creating their models. 


Let's go through an example.

### Add create a report option to the Sick Sicker App

#### Modify the UI and Server in the app.R file

First, we can start with modifying the app.R file. We can define a download button at the UI and add a downloadhandler() at server side as follows:

```{r, eval=FALSE}
#---  TO INSERT in the UI in the app.R  ---#
# Create a button to request a report output, this can be inserted after the action button in the app.R of Sick Sicker 
   downloadButton("myreport", "Create a Report")
```


```{r, eval=FALSE}
#---  TO INSERT BEFORE THE END OF THE SERVER in the app.R  ---#

# Name of the output should match the Id (first) parameter of the download button 
# e.g myreport here.
  output$myreport <- downloadHandler( 
    # For a PDF output, modify the extension to ".pdf"
    filename = "report.html",
    content = function(file) {
      # Create a function which will copy the report in a temporary directory before processing it.
      tempReport <- file.path(tempdir(), "report.Rmd") # tempdir() function creates a temporary directory
      file.copy("report.Rmd", tempReport, overwrite = TRUE) # Copies the report.Rmd to the temporary directory
      
      # Set up parameters to be written to the rmd document in a list
      # Here the user modified parameters are Cost of Treatment, 
      # number of simulations to run and the initial age 
      params <- list(n = input$SI_c_Trt, m = input$SI_n_sim, x =input$SI_n_age_init)
      
      # To knit the report.Rmd with `params` list, and eval it in a
      # child of the global environment (this isolates the code in the document
      # from the code in this app).
      # This provides better code isolation and makes things easier to debug.
      rmarkdown::render(tempReport, output_file = file,
                        params = params,
                        envir = new.env(parent = globalenv())
      )
    }  
)
```



In the next step, We will pass the parameters specified above by the`params` list to the Rmd document.

#### Setting up the report.Rmd
When we run the modified app.R it will generate the list `params` at the server side and now we have to tell the rmd document how to access those variables. Parameters that will be passed from Shiny should be specified in the YAML header (top section between three horizontal lines `---` of the rmd document and should be with two indentation from the beginning of a new line and initally set to `NA` as below.

```{r, eval=FALSE}
---
title: "Dynamic report"
output: html_document
params:
  n: NA
  m: NA
  x: NA
---
```

Those `params` values will be updated after the user changes the corresponding variables in the shiny app and run a model. 

We can create a table with the model parameter values by using an inline R code. In this case, Parameters should be wrapped the inside an ``` `r ` ``` expression.

Before knitting:

```{r eval=FALSE}
|  Parameter  | Set value 
|-------------|-----------
| Cost of treatment | `r params$n` 
| Number of PSA runs | `r params$m`
| Initial age | `r params$x`

```


After knitting:

|  Parameter  | Set value 
|-------------|-----------
| Cost of treatment | `params$n`
| Number of PSA runs | `params$m`
| Initial age | `params$x`

Now, after setting up which user input values to include in the report, we need to include the resulting plots and tables.

When users enter their parameters and click Run in the Sick Sicker App `f_wrapper` function will create the model `df_model_res`. To be able to pass the same model to the report.Rmd we can add an extra step in the app.R so a local copy of the model is saved. 

This pre-saved model will be used to render the QALY table and plot in the report. 

**Add an extra step in the app.R:**

```{r, eval=FALSE}
# To insert in the app.R after df_model_res
# Create a reactive variable which will be updated when a user clicks run button
 df_model_save <- eventReactive(input$run_model, { df_model_res})
# The model will be saved in the same folder as the app
 saveRDS(df_model_save(),"df_model_res.rds")
```

With the above code the model will be saved whenever users click run model. We can utilize this saved model in report.rmd to create the QALY table and the cost effectiveness plot.

**Create the QALY table:** 

```{r, eval=FALSE}
library(magrittr)
# Add to report.rmd
# Read the saved PSA model, to create the table and the plot 
df_model_res <- readRDS("~/workingdirectory/rforhealthcare/sicksicker/df_model_res.rds")

df_res_table <- data.frame( # create dataframe
                     
                     Option =  c("Treatment","No Treatment"), 
                     
                     QALYs  =  c(mean(df_model_res$QALY_Trt),mean(df_model_res$QALY_NoTrt)),
                     
                     Costs  =  c(mean(df_model_res$Cost_Trt),mean(df_model_res$Cost_NoTrt)),
                     
                     Inc.QALYs = c(mean(df_model_res$QALY_Trt) - mean(df_model_res$QALY_NoTrt),NA),
                     
                     Inc.Costs = c(mean(df_model_res$Cost_Trt) - mean(df_model_res$Cost_NoTrt),NA),
                     
                     ICER = c(mean(df_model_res$ICER),NA)
                  )
# kableExtra package produces a better table format.
library(kableExtra)

kable(df_res_table) %>%
kable_styling(bootstrap_options = c("striped", "hover"))  # Print the resulting table
```

**To create the cost effectiveness plot:**

```{r, eval=FALSE}
# create cost effectiveness plot

df_model_res$inc_C <- df_model_res$Cost_Trt - df_model_res$Cost_NoTrt
                   df_model_res$inc_Q <- df_model_res$QALY_Trt - df_model_res$QALY_NoTrt
                   
                   plot(x = df_model_res$inc_Q, # x axis incremental QALYS
                        y = df_model_res$inc_C, # y axis incremental Costs
                        # label axes
                        xlab = "Incremental QALYs", 
                        ylab = "Incremental Costs", 
                        
                        # set xlimits and ylimits for plot.
                        xlim = c(min(df_model_res$inc_Q,df_model_res$inc_Q*-1),
                                 max(df_model_res$inc_Q,df_model_res$inc_Q*-1)),
                        ylim = c(min(df_model_res$inc_C,df_model_res$inc_C*-1),
                                 max(df_model_res$inc_C,df_model_res$inc_C*-1)),
                        # include y and y axis lines.
                        abline(h = 0,v=0)
                   )
```

The created HTML file will already include all plots and images embedded directly. There is no additional step e.g. for handling separate image files. 

You can find the modified app.R and the report.Rmd here (TO be done: add a github link)

Now, let's have a look at the final report.

![Dynamic Report](_bookdown_files/bookdown-demo_files/figures-shiny/dynamic_report.PNG)

Additional parameters to be written on the report can be added to the YAML header of the report.rmd.  For example, you can include the time when the report is created. Use !r before an R expression to pass it as a parameter to the report (e.g date: !r Sys.Date()).


### Importing as PDF or Word

Change the filename argument of downloadHandler() to "report.pdf" or "report.doc. Use output: pdf_document or output: word_document respectively in the YAML header of the Rmd file to get an pdf or word output. Creating a pdf report will also require `pdflatex` to be installed on your system.

The rmarkdown::render() function has many options to control the processing and output. See the rmarkdown website to learn more <https://rmarkdown.rstudio.com/>.

### Sending Scheduled Automated reports to be emailed to you from Shiny apps

Can be added.


## Modify the appearance of Shiny apps

Shiny is a web application, thus to be interpreted by web browsers it uses the languages of the web, such as HTML (HyperText Markup Language) which is the basic building block of Web pages, CSS (Cascading Styles Sheets) which adds a wide variety of formating options to style HTML elements and Javascript which allows for dynamic options of a web page such as user interactivity.  

Shiny UI Code is a set of R functions that output an HTML document along with the CSS and Javascript code that is included. 

Thus, understanding some basics of HTML, CSS and Javascript will allow us to customize our apps.


### Basic introduction to HTML and CSS 

An HTML document is a collection of tags which is interpreted by the web browser step by step.

A minimal HTML structure may be defined as follows:

```{r, eval=FALSE}
<!DOCTYPE HTML> # Declaring that the document is an HTML document
<html>   # Wraps the entire file

  <head>  # Wraps the meta info
  
      <!-- head content here -->
  
  </head>
  
  
  <body> # Wraps the content of the page
  
     <p> Hello World! </p>  # paragraph tag
  
  </body>

</html>
```

The head part `<head>...</head>` can contain the dependencies like styles and Javascript as well as meta data (e.g. information about the page or google analytics info) and does not appear on the page visually. 
`<body>...</body>` contains the main content which is displayed on the screen.

Each of the `<.>` is called a tag. Text are wrapped between an opening and closing tag.
The opening and closing tag together forms an element. For example, `<p>.</p>` is called the "p element".


#### Tag attributes

HTML Tags can also include attributes. Attributes are name/value pair and the general syntax is

```{r, eval=FALSE}
<p name="value"> ... </p>
```

The two most common attributes are class and id attributes. Class names does not have be unique and are used to target group of elements, however, ids should be unique for each element. 

Here is an example:

"div" is a general markup for a block of text.

```{r, eval=FALSE}
<div class="items" id="myitem-1"></div>
<div class="items" id="myitem-2"></div> 
<div class="items" id="myitem-3"></div> 
```

Here we have multiple elements with the same HTML tag in your document. If we want to stylize each of them differently we can use the id attribute to target them.

For instance, following CSS code will turn the text inside "myitem-" elements to the corresponding colors. To specify the Id `#` syntax is used before the id name.

```{r, eval=FALSE}
#myitem-1 {
  color: blue;
}

#myitem-2 {
  color: red;
}

#myitem-3 {
  color: yellow;
}
```


And if we want to target all three elements we can use the class attribute that each of these div tags share. To add style to all elements that are part of a particular class, you preface the class name with a period (.) in the CSS. For example to make all div tags with the `items` class blue:

```{r, eval=FALSE}
div.items {
  color: blue;
}
```

Thus, they are extensively used by CSS and Javascript to target an element for applying a custom style or giving a dynamic functionality.


This is good to keep in mind since Shiny uses under the hood names and classes to specifically target an element or group of elements respectively, e.g. changing colors of some button/s.


### Adding HTML & CSS Styles into an R Shiny App

We can wrap any text that we want to modify in a Shiny app with an HTML tag. h3 tag defines a third level header in HTML. To make a text to have h3 style we need to wrap it inside the tag function e.g. h3() (or tags$h3()). We can use the style argument inside and HTML tag function to modify the style with CSS. We can include here as many as we would like CSS elements. Multiple arguments need to be separated with a `;`.

Let's modify the titlePanel of our Sick Sicker app.

Current version:

```{r, eval=FALSE}
library(shiny)
titlePanel("Sick Sicker Model in Shiny")
```

Apply HTML h3 heading style:
```{r}
titlePanel( h3("Sick Sicker Model in Shiny"))
```

Apply HTML h3 heading style with `tags$` syntax:
```{r}
titlePanel(tags$h3("Sick Sicker Model in Shiny"))
```

We can apply CSS style to any text wrapped with an HTML tag by using the style argument as in the following:
```{r}
titlePanel( h3("Sick Sicker Model in Shiny", style = "color: #48ca3b;"))

```

To add multiple CSS arguments, separate each argument with `;`:
```{r}
titlePanel(h3("Sick Sicker Model in Shiny", style = "color: #48ca3b; background-color: #000000; text-align: center;"))
```

Adding an hyperlink: `<a> </a>` tag allows for adding a hyperlink to a text. `<a>` tag has other attributes such as href to point the address of the link. Set external=TRUE if it the link points to an external website.

```{r}
h2(a("An awesome tutorial", href = "https://r-hta.org/tutorial/markov_models_shiny/" , external=TRUE))
```


There are helper functions in Shiny that can call their equivalent HTML tags directly such as h3() we used above. Those are: 

a(): creates a link
br(): inserts a linebreak
code(): for inserting a computer code
div(): for a block of text
em(): for italicing text
h1() to h6(): for creating headers
img(): allows to insert an Image
p(): for paragraph
pre(): for preformatted text
span(): for an inline text
strong(): for bold text

For other HTML tags user has to use `tags$` syntax before the HTML tag. To see all the tags available to Shiny simply type `names(tags)` in R console. You can learn more about the most common tags in [Shiny HTML tags glossary.](https://shiny.rstudio.com/articles/tag-glossary.html)

### Adding Raw HTML to Shiny

It is also possible to add raw HTML to Shiny UI. HTML() function takes a character string and returns it as HTML (a special class of object in Shiny).

With Shiny h3() function:
```{r, eval = FALSE}
titlePanel( h3("Sick Sicker Model in Shiny"))
```

With Raw HTML:
```{r, eval = FALSE}
titlePanel( HTML("<h3>Sick Sicker Model in Shiny</h3>"))
```

### More Customization examples

I will add here more examples 
e.g. modify appearance of boxes etc.

### Shiny packages to modify the themes

Can be included.

## How to wrap up Shiny apps using electron


## Sharing a Shiny app with Docker

You can share your Shiny apps easily by hosting them on commercial platforms such as [RStudio](https://shiny.rstudio.com/deploy/) or with some additional effort by setting up and managing your own servers. When you don't have access to a Shiny server or you have sensitive data that you don't want to store online you have another alternative Docker. 

Docker allows people who does not have programming skills to run your app in their local computers. You can make your health economic models shareable as well as ensure them to work in any other computer (e.g windows, mac or linux). 
Docker enables you to pack in all the components required to run your Shiny or any other application, with all code, dependencies and the server. 

![Docker vs VM (source: www.docker.com)](_bookdown_files/bookdown-demo_files/figures-shiny/docker_vs_vm.PNG)

Unlike a virtual machine, in Docker the containers running share the host OS kernel. A Dockerized application is just a process that runs on your system. It doesn't require running a Hypervisor (such as VMWare or VirtualBox). Thus, Docker uses much less resources of the operating system. 

Two concepts is very confusing at the beginning for people who don't know about Docker. So let's explain them.

**Docker Image vs Docker Container**

Docker Image:

A Docker image is a non-changeable file containing libraries, source code, tools and other files needed to run applications.

Docker Container: 

Container is a running instance of a docker image. In context of a Shiny app, this will be the running instance of a Shiny app. With an anology, we can think of Docker Image as a digital photograph and Docker Container is a print of this digital image.



### How to Dockerize your shinyApp?

First, you need to [download](https://www.docker.com/get-started) Docker Desktop (either for mac or windows) and go through a tutorial [here.](https://ropenscilabs.github.io/r-docker-tutorial/) After installing Docker, you will need to set up a folder where your app and a couple of other files we need will be stored.

### Example: Dockerize the Sick Sicker app  

Let's go through an example to see how to do it. The process starts with creating a Dockerfile. It is a text document that contains all the instructions for Docker to pack all the necessary components for running your applications. 
The following is the Dockerfile we will use to create a Docker image of the Sick Sicker app. This file should be named as `Dockerfile`.

```{r eval=FALSE, include=TRUE}
# Install R version 3.6
FROM r-base:3.6.0

# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
    sudo \
    gdebi-core \
    pandoc \
    pandoc-citeproc \
    libcurl4-gnutls-dev \
    libcairo2-dev/unstable \
    libxt-dev \
    libssl-dev

# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
    VERSION=$(cat version.txt)  && \
    wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
    gdebi -n ss-latest.deb && \
    rm -f version.txt ss-latest.deb

# Install R packages that are required for the app
# Specify further packages if you need!
RUN R -e "install.packages('shiny', repos='http://cran.rstudio.com/')"
RUN R -e "install.packages('truncnorm', repos='http://cran.rstudio.com/')"

# Copy configuration files into the Docker image
COPY shiny-server.conf  /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/

# Make the ShinyApp available at port 3838
EXPOSE 3838

# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh

CMD ["/usr/bin/shiny-server.sh"]
```

Let's break down these set of commands to look at what they are doing. 


This will install the R version 3.6.0 image file. 

```{r eval=FALSE, include=TRUE}
FROM r-base:3.6.0
```

Below commands will install all the necessary linux components required to run a Shiny Server. 

```{r eval=FALSE, include=TRUE}
RUN apt-get update && apt-get install -y \
    sudo \
    gdebi-core \
    pandoc \
    pandoc-citeproc \
    libcurl4-gnutls-dev \
    libcairo2-dev/unstable \
    libxt-dev \
    libssl-dev
```

Then, we will run some other commands to install Shiny Server.

```{r eval=FALSE, include=TRUE}
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
    VERSION=$(cat version.txt)  && \
    wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
    gdebi -n ss-latest.deb && \
    rm -f version.txt ss-latest.deb
```

Following lines will instruct Docker to download the specified R packages. In our case, we only need shiny and truncnorm packages.

```{r eval=FALSE, include=TRUE}
RUN R -e "install.packages('shiny', repos='http://cran.rstudio.com/')"
RUN R -e "install.packages('truncnorm', repos='http://cran.rstudio.com/')"
```

If you want always to use a fixed version of a package (e.g Shiny) you can set this by pointing an archived .tar file as follows:

```{r, eval=FALSE}
RUN R -e "install.packages(pkgs='https://cran.r-project.org/src/contrib/Archive/shiny/shiny_1.4.0.1.tar.gz',repos=NULL,type='source')"
```

Next lines will copy the shiny-server configuration file and the app/ folder to the ubuntu shiny-server folder where the app will run (e.g. /etc/shiny-server/ folder).

```{r eval=FALSE, include=TRUE}
COPY shiny-server.conf  /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
```


#### Shiny-server configuration file

This file sets up the user for the shiny-server folder so that the app can run. Copy this file into the same directory as the Docker file and name as shiny-server.conf. 

```{r eval=FALSE}
# Define the user we should use when spawning R Shiny processes
run_as shiny;

# Define a top-level server which will listen on a port
server {
  # Instruct this server to listen on port 3838
  listen 3838;

  # Define the location available at the base URL
  location / {

    # Run this location in 'site_dir' mode, which hosts the entire directory
    # tree at '/srv/shiny-server'
    site_dir /srv/shiny-server;
    
    # Define where we should put the log files for this location
    log_dir /var/log/shiny-server;
    
    # Should we list the contents of a (non-Shiny-App) directory when the user 
    # visits the corresponding URL?
    directory_index on;
  }
}

```

Create an additional file with the following name Shiny-server.sh and place in the same folder as the Dockerfile.

```{r, eval=FALSE}
#!/bin/sh

# Make sure the directory for individual app logs exists
mkdir -p /var/log/shiny-server
chown shiny.shiny /var/log/shiny-server

exec shiny-server >> /var/log/shiny-server.log 2>&1
```

The commands on this file gives shiny user ownership of the shiny-server folder so that the user can run the app.

At this step the Docker folder should look like this. 

![ScreenPrint of Sick Sicker Docker folder](/_bookdown_files/bookdown-demo_files/figures-shiny/docker_sicksicker_screenclip.PNG)


**Build a docker image of your app**

Then we will move to Command Prompt (Windows) or Mac (Terminal) and change directory to the folder where our Dockerfile is and run the following commands:

```{r, eval= FALSE}
# This will build the docker image
docker build -t my-shiny-app .
```

You can use the name you wish by switching it with `my-shiny-app`. This will begin the creation of the Docker image using your Dockerfile as instructions. At this step, Docker will download all the necessary components for your app and build an image that you can already share with other users. 


**Create a container**

```{r, eval= FALSE}
# This will start a container running an instance of the docker image
docker run --rm -p 3838:3838 my-shiny-app
```

This will create a container of your app from the Docker image file. You can switch to Docker desktop app and click Open in Browser button to view it on the browser at port 3838.


![Docker Desktop ready for viewing the app](/_bookdown_files/bookdown-demo_files/figures-shiny/dockerdesktop_clickrun.PNG)

Now, Sick Sicker app is running on local host 3838.

![Sick Sicker running on localhost](/_bookdown_files/bookdown-demo_files/figures-shiny/sicksicker_localhost.PNG)

Once you have created a Docker Image of a Shiny app, you can send that image to someone else as a .tar file or through the Docker hub. They will be able run the application exactly like yours. 

If an application works on your local computer, this means it will work on any other machine that has Docker Desktop installed. This ensures any user can run your Shiny application without a need install programs or libraries. They just need to have Docker Desktop installed and run two lines of script on a terminal. Docker images are immutable, this means once created, they will always work. 


### Saving Docker image

After building the Docker image with `docker build -t my-shiny-app .` command we can also save a .tar file of the image for sharing with other people. The following fill create a .tar file of the Docker image inside our original Docker folder.

```{r,eval=FALSE}
docker save my-shiny-app > myshinyapp.tar
```

This will create the myshinyapp.tar in the same folder we started with. Now, we can share the myshinyapp.tar with someone else and he/she will be able to run the app easily independent of the operating system they use. Here, the total size of the myshinyapp.tar file was about 1.7GB.

To start an application from this .tar file we need to load the Docker image in terminal as follows:
```{r, eval=FALSE}
docker load < myshinyapp.tar
# And start the container
docker run --rm -p 3838:3838 my-shiny-app
```

The Docker Image folder we used can be found here (links to be added).

### How to share Docker Images via Docker hub

To be added. 

### Other Useful Shiny packages

To be added.