Scaling Shiny Applications through Containerization

Preface

How often have you as an analyst wished you could create an application and easily share your analysis, reports and results with others - be it clients, team members, business leaders etc., and in an interactive manner? Yes you can and one need not be an html expert or a web developer. Here comes Shiny. One can whip up a simple Shiny application to share analytic insights over the web. Of course, having knowledge and expertise in html and web development will help take your shiny application to another level.

What is Shiny?

Shiny is an R and Python library that enables one to create interactive web applications. These web applications can be anything from an interactive dashboard, predictive model results, interactive plots and a myriad other things; and they are user-centric without imposing complex intricacies, additional software etc., on the user.

The Shiny ecosystem comes with a rich set of developmental functions for creating priceless web applications and the following link is an excellent resource providing a copious compendium of Shiny info.

In this blog a gentle guidance with a practical illustration of prototyping a Shiny application and scaling the same will be showcased.

• The first stage will illustrate the prototype developed and launched on a local machine.
• The second stage will take the prototype application and launch it on the web using the Shinyapps.io platform.
• The third stage will take the prototype and scale it by containerizing the application on Docker.

In a previous blog on Containerization of R models, the Boston Housing Market data from the U.S. census service(MASS library) was utilized to illustrate how to develop Plumber APIs, containerize & serve the API/model results and make bulk predictions.
In order to maintain continuity of the data theme, the same Boston Housing Market data will be used here to build and deploy a Shiny application.

The model can be served/called to the Shiny application by one of the following approaches

• A pre-built plumber API can be served/fed to the shiny app and predictions can be made based on the user inputs.
• The saved model can be called into the shiny application on the server side and used in making predictions based on the user inputs.
  In the ensuing shiny demos - the second approach will be utilized and expanded upon.

Dev & Deploy App - Local Machine

The Boston Housing Market data gathered from the U.S Census service (MASS library) is called upon to prototype/build the Shiny application and scale/containerize the same.

The shiny app will be first prototyped on a local machine.

library(MASS)
library(randomForest)
library(dplyr)
library(yaml)
glimpse(Boston)

1

MASS library is called for importing the Boston Housing data.

Rows: 506
Columns: 14
$ crim    <dbl> 0.00632, 0.02731, 0.02729, 0.03237, 0.06905, 0.02985, 0.08829,…
$ zn      <dbl> 18.0, 0.0, 0.0, 0.0, 0.0, 0.0, 12.5, 12.5, 12.5, 12.5, 12.5, 1…
$ indus   <dbl> 2.31, 7.07, 7.07, 2.18, 2.18, 2.18, 7.87, 7.87, 7.87, 7.87, 7.…
$ chas    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ nox     <dbl> 0.538, 0.469, 0.469, 0.458, 0.458, 0.458, 0.524, 0.524, 0.524,…
$ rm      <dbl> 6.575, 6.421, 7.185, 6.998, 7.147, 6.430, 6.012, 6.172, 5.631,…
$ age     <dbl> 65.2, 78.9, 61.1, 45.8, 54.2, 58.7, 66.6, 96.1, 100.0, 85.9, 9…
$ dis     <dbl> 4.0900, 4.9671, 4.9671, 6.0622, 6.0622, 6.0622, 5.5605, 5.9505…
$ rad     <int> 1, 2, 2, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4,…
$ tax     <dbl> 296, 242, 242, 222, 222, 222, 311, 311, 311, 311, 311, 311, 31…
$ ptratio <dbl> 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15…
$ black   <dbl> 396.90, 396.90, 392.83, 394.63, 396.90, 394.12, 395.60, 396.90…
$ lstat   <dbl> 4.98, 9.14, 4.03, 2.94, 5.33, 5.21, 12.43, 19.15, 29.93, 17.10…
$ medv    <dbl> 24.0, 21.6, 34.7, 33.4, 36.2, 28.7, 22.9, 27.1, 16.5, 18.9, 15…

The tools used in these illustrations and app development are primarily RStudio and VS Code in some areas. RStudio is well suited for your R Shiny applications but you can also use VS code for your R and Python Shiny applications.

The first step to is to create a project/folder where all the necessary files to execute the shiny application are reposited.

A random forest model is run for use in the shiny application.

## Build a Random Forest Model 
boston_rf_model <- randomForest(medv ~., data = Boston, mtry =3, ntree = 500, importance = TRUE)

The model is saved in the same project/folder where the shiny app (app.R) will be created and placed.

## save the model
saveRDS(boston_rf_model, "boston_rf_model.rds")

Next, the RMSE (Root Mean Square Error) is computed which will also be saved in the same project/folder as the boston_rf_model and the Shiny app.R file. The RMSE will be used to compute the lower and upper limit of the confidence interval for the predicted value of the Median Housing Value based on the user’s input values.

## execute prediction on the training data 
pred_medv <- predict(boston_rf_model, Boston)

## compute the RMSE for the predicted /actual median housing values
bost_rmse <- sqrt(mean((pred_medv - Boston$medv)^2))

## save the rmse value as an R file
saveRDS(bost_rmse, "bost_rmse.rds")
# print(bost_rmse)

Let us now dive into the building of the app and some high level details on the components.

Every shiny application has three major components:

• The UI (user interface) function.
• The Server function
• The execution (runApp) function that combines the UI and Server function to execute the application.

The shiny app created and demoed here (based on the Boston Housing Market data) is fairly simple and straight-forward with user friendliness and efficiency being prioritized.

The UI component and server component are broken into two chunks of code in this document for ease of reading and flow. Another final chunk of code has the runApp for execution which is muted here (within the blog).

When executing the application these two components along with the ‘runApp’ function(that combines the UI & Server functions) can all be embedded in one chunk and saved as app.R.

Highlights and salient features of the app

• The necessary libraries are loaded.

• The UI component has all the features that the user interfaces with - namely the user’s inputs, and the outputs rendered for the user when the server function is executed.

• In addition to the user inputs for the model variables, another salient input that is necessary to trigger reaction from the server function is the ‘actionButton’. The user would need to hit/execute the ‘goButton’ after providing the necessary inputs and only then the server function components will be activated.

• The loading of an image in the user interface of the app - to provide visual effects to the front-end.

• The outputs rendered by the server function - a table of the input values, the predicted median housing value and the lower and upper confidence level once the server function is triggered by the ‘actionButton’. These generated outputs will update only when and each time the ‘actionButton’ is executed by the user.

• One can use various themes from readily available libraries such as ‘shinythemes’ (utilized here), bootstrap - bslib etc., or create one’s own customized themes to provide visual enhancement to the user interface.

• Styling effects can be incorporated and these can be added either directly to html tags, or as css to html headers (utilized here) or through external styling sheets.

  • For small apps it makes sense to have inline styling or internal styling, but having a styling sheet that is separate from the app especially where the app is large and complex is prudent as it gives the core app a clean look and also makes changes and updates to styles transparent and global.

• The server function does the “back-end” operations.

• The model and RMSE are loaded and ready to go to work.

• The ‘eventReactive’ actions are performed based on the trigger from the UI when user hits the ‘actionButton’. And once these are executed the ensuing ‘Reactive’ functions perform their actions such as calculating the prediction, the lower and upper confidence level using the model and RMSE.

• The output needed in the user-interface /front-end are rendered as a table or text output of prediction and confidence levels.; these are subsequently called into the UI as outputs.

library(shiny)
library(shinythemes)
library(randomForest)

# Define User Interface Logic/Scheme ---

ui <-fluidPage(
        theme = shinytheme("darkly"),
      # Application Title
        titlePanel(h3(strong(div("Predicting Median Value of Housing in the Boston Market (Using Random Forest)", style = "color: bisque", align = "center"))), windowTitle = "Predicting Median Value of Housing in the Boston Market"),
        br(),
        fluidRow(
        #Column 1 Inputs
          column(width = 4, (h5(strong(div("Input Variables", offset = 0, style = "color: chocolate", align = "center")))),
            br(),
            sliderInput("crim",
                     "Crime Rate (crim)",
                      min = 0.0000,
                      max = 90.0,
                      value = 0.0063,
                      step = 0.001,
                      round = F,
                      width = '95%'),
                    
            sliderInput("zn",
                      "Zoned Land > 25K sq ft (zn)",
                       min = 0.0,
                       max = 100.0,
                       value = 18.0,
                       step = 3.5,
                       round = F,
                       width = '95%'), 
          
            sliderInput("indus",
                       "Propn of Non-Retail Business Acres (indus)",
                        min = 0.40,
                        max = 28.00,
                        value = 2.31,
                        step = 0.15,
                        round = F,
                        width = '95%'),
                         
            selectInput("chas",
                       "Charles River Dummy - Track Bounds River No/Yes (chas)",
                        choices=list(0, 1), 
                        selected = 0,
                        width = '95%'),
        
            sliderInput("nox",
                       "nitrogen oxides cocentration (nox)",
                        min = 0.201,
                        max = 0.901,
                        value = 0.538,
                        step = 0.051,
                        round = F,
                        width = '95%'),
                         
            sliderInput("rm",
                       "Avg no. of rooms (rm)",
                       min = 3.000,
                       max = 9.000,
                       value = 6.575 ,
                       step = 0.499,
                       round = F,
                       width = '95%'),
          
            sliderInput("age",
                      "Propn of Owner Occupied Units built prior 1940 (age)",
                      min = 2.00,
                      max = 100.00,
                      value = 65.2,
                      step = 2,
                      round = F,
                      width = '95%'),
            ),
        #Column 2 Inputs
          column(width = 4, (h5(strong(div("Input Variables", style = "color: chocolate", align = "center")))),
            br(),
            sliderInput("dis",
                  "Wtd Distance to 5 Boston employment centers (dis)",
                  min = 1.00,
                  max = 13.00,
                  value = 4.10,
                  step = 0.2,
                  round = F,
                  width = '95%'),
      
            selectInput("rad",
                  "Accessibility to radial highways (rad)",
                  choices=list(1, 2,3, 4, 5, 6, 7, 8, 24), 
                  selected = 1,
                  width = '95%'),
                  
            sliderInput("tax",
                  "Property Tax-rate per $10k (tax)",
                  min = 100.0,
                  max = 750.0,
                  value = 296.0,
                  step = 20.0,
                  round = F,
                  width = '95%'),
      
            sliderInput("ptratio",
                  "Pupil-Teacher Ratio (ptratio)",
                  min = 10.0,
                  max = 25.0,
                  value = 15.3,
                  step = 0.5,
                  round = F,
                  width = '95%'),
      
            sliderInput("black",
                  "Wtd Propn of Blacks (black)",
                  min = 0.30,
                  max = 400.0,
                  value = 396.9,
                  step = 2.5,
                  round = F,
                  width = '95%'),
      
            sliderInput("lstat",
                  "% of Lower status of the popln (lstat)",
                  min = 1.01,
                  max = 40.01,
                  value = 4.98,
                  step = 0.51,
                  round = F,
                  width = '95%'),
                  
          actionButton("goButton", "Hit-Go!", class = "btn-success"),
          br(),
          br(),
          h5(strong("The predicted Median Housing Value(Year-1978) of your Boston Home(in $1000s) is :", style = "color: goldenrod",
          span(textOutput("pred_medv"), style = "color: firebrick; font-size: 15px", align="right"))), 
      
          h6("The Lower Level Conf. Interval and Upper Level Conf. Interval at the 95% Level for Predicted median value are:", style = "color: goldenrod"),
          span(textOutput("lower_CI"), textOutput("upper_CI"), style = "color: dodgerblue; font-size: 14px", align="right")
         ),
   
      #Column 3 - Info
        column(width = 4, (h5(strong(div("Hello Boston!", style = "color: chocolate", align = "center")))),
          br(),
          HTML('<center><img src = "DreamHouse.jpg", width = "auto", height = "400" ></left>'),
          br(),
          br(),
          tableOutput("values")
        )
    )
)

# Define server logic ----
server <- function(input, output) {
  
# load the model ---
# load the RMSE  ---
    model <- readRDS("boston_rf_model.rds")
    rmse <- readRDS("bost_rmse.rds")
  
# Create a data-frame for rendering the selected input variables and their values as a table
# It is event reactive and updates only when the action-button has been executed by the user
    
  Input_Values <- eventReactive(input$goButton, {
  data.frame(
    Predictors = c("CrimeRate",
                   "Propn of Zoned Land",
                   "Propn of Non-Retail Business",
                   "Charles River Tract",
                   "Nitrogen Oxide conc",
                   "Avg no. of rooms",
                   "Propn- Owner occup older homes",
                   "Distance(Wtd) to empl centers",
                   "Access to highways",
                   "Tax-Rate per $10k",
                   "Pupil-Teacher-Ratio",
                   "Propn of Blacks",
                   "Lower status popln"),
    Value = as.character(c(input$crim,
                           input$zn,
                           input$indus,
                           input$chas,
                           input$nox,
                           input$rm,
                           input$age,
                           input$dis,
                           input$rad,
                           input$tax,
                           input$ptratio,
                           input$black,
                           input$lstat)),
    stringsAsFactors = FALSE)
})

# The following code takes the values and renders it into a table from a call in the UI

  output$values <- renderTable({
  Input_Values()
})

# The following code creates a dataframe for feeding into the saved Random forest model
# It is event reactive and updates only when the action-button has been executed by the user
  
  input_df <- eventReactive(input$goButton,{
              data.frame(crim = input$crim,
                         zn = input$zn,
                         indus = input$indus,
                         chas = as.double(input$chas),
                         nox = input$nox,
                         rm = input$rm,
                         age = input$age,
                         dis = input$dis,
                         rad = as.double(input$rad),
                         tax = input$tax,
                         ptratio = input$ptratio,
                         black = input$black,
                         lstat = input$lstat)
    
})

# A reactive expression that computes the prediction based on the input dataframe 
pred_medv <- reactive({
  round(predict(model, input_df()),3)
  })

# Computes the lower confidence level of the predicted value

lower_CI <- reactive({
  round((predict(model, input_df()) - (1.96*rmse)), 3)
})

# Computes the upper confidence level of the predicted value
upper_CI <- reactive({
  round((predict(model, input_df()) + (1.96*rmse)),3)
})

# outputs texts to be called into the UI interface.

output$lower_CI <- renderText({paste0("Lower Limit CI:  ", lower_CI())})

output$upper_CI <- renderText({paste("Upper Limit CI:   ", upper_CI())})

output$pred_medv <- renderText({pred_medv()})
}

# Run the app ----
## The below line of code can be executable after removing the hashtag. The hashtag
## here ensures that the app is not executable within a quarto doc.

#shinyApp(ui = ui, server = server)

Figure 1 shows all the files in the current project/folder that are necessary to execute the Shiny application on the local machine.

In addition to all the files that were discussed above there is another folder - ‘www’

• This folder will contain any images that you wish to incorporate in your application.
• This is also the folder where a customized styling sheet will be placed in the event you have one that you will be calling into your application.

The highlighted ‘Run App’ button shown below will run your application when executed and your application will show up on your machine’s local browser.

Figure 1 - List of Files for Shiny app generation (local machine)

Figure 1: List of Files for the Shiny App Generation - Local Machine

The screenshots of the application output are shown in Figure 2. The first visual (a) shows the app awaiting the user’s input and execution of actionButton. The second visual (b) shows the input and output results from user interaction.

Figure 2 - Shiny Output (Local Machine)

(a) shiny-app-local

(b) shiny-app-local

Figure 2: Shiny Output (Local Machine)

Deploy App - Shinyapps.io

It would be great and often times vital to take a prototype Shiny application on a local machine and be able to share the same with others/ provide a quick demo (sans any proprietary info); One can easily accomplish that by deploying them on Shinyapps.io in no time.

The next few lines and figures are around setting up a shinyapps.io account and steps to host one’s app on that platform.

If you would rather skip this section and go straight to the hosted application, scroll down to the Try-the-Demo! button/ link shown below.

The shinyapps.io website is a great resource(provided by Posit) for hosting your shiny demos and it has a free option as long as one’s resource utilization and the number of apps hosted are within the set thresholds.

1. Have the project/folder with your shiny app open in your RStudio and run the app locally.

2. Next, open a new file (while still within the same project/folder); install and load the library(rsconnect)

   ##install the rsconnect package
   #install.packages(rsconnect)
   ##load the rsconnect package
   #library(rsconnect)

3. You can create a shinyapps.io account if you don’t already have one using the following link - ShinyApps.io sign up page.

4. The following link provides meticulous step-by-step instructions & visuals on configuring and hosting your app on shinyapps.io - Shiny-apps account configuration.
   Nevertheless, the steps shown below highlights them, providing continuity to the document narrative.

5. Once you have created account or already have one, log into your account and go to your dashboard. The below figure shows you how the screen would look like on shinyapps.io.

6. Figure 3 - Screenshot of the shinyapps.io dashboard

   

   Figure 3: Shinyapps.io dashboard page

7. If you just created an account, the dashboard will be empty; Click the profile pic and a drop-down menu will show up and there will be a ‘Tokens’ option listed in that drop-down menu. Click on that. Your screen will now look like the below figure.

   Figure 4 - Token Screen on Shinyapps.io

   

   Figure 4: Shinyapps.io-Token Screen

8. If you just created an account, this page will be blank - click ‘Add token’, OR you may already have a token. Once you have your screen/account populated with a token click on ‘Show’

9. You will see something similar to the figure shown below. Click the ‘Copy to clipboard’ and that will result in a pop-up showing up. Copy the details and go over to your Rstudio.

   Figure 5 - rsconnect set Account Info for App Launch

   

   Figure 5: rsconnect-Token-Screen

10. Go to Rstudio and paste the details you copied from your shinyapps.io account and paste it in the Console section and hit Enter. Your screen should look similar to the one shown below.

    Figure 5 - RSconnect token configuration and App hosting

    

    Figure 6: Shinyapps token details on your console

11. Once you hit enter and the command prompt shows up after execution, run the deployApp() command shown in the above screenshot. This will go through several steps/stages and load all necessary files to your shinyapps.io account and launch your app on the web.
    Note: - If you encounter any errors in the process and shinyapps.io fails to launch your app, go to the logs file on your account and look for the reason/s. Often times things such as file naming convention etc., can throw off the hosting of the app.

12. Another point of note is that once you have hosted the app, if you now look at your project/folder where you have all the relevant files, app files etc., a new folder would have been created/showed up in the process and named ‘rsconnect’ as shown below. This folder will contain info about your app such as the web-link to the app and other sundry details.

Figure 7 - Files for launching app on Shinyapps.io as well the newly created rsconnect folder

Figure 7: List of Files including rsconnect folder for launching app on shinyapps.io

Now to the fun part - where you can actually interact with the app that has been discussed above and hosted on Shinyapps.io.

Click the highlighted link below to take you to the app and try it out!

Try-the-Demo!

Deploy App - Docker

In order to productionalize and scale your app within the walled garden of a corporate entity, it needs to be containerized and served on the cloud.

The following steps provide a sequence of tasks/processes towards containerization of your app:

• Be sure to have Docker Desktop up and running on the local machine before any Docker commands are executed. You can download the Docker Desktop using the following link - Docker Desktop Installation Guide.
• The dockerfile for Shiny applications will be configured slightly different from what might seem intuitive and used for other applications such as developing an API.
  • If one took the shiny-app files and folders as used on a local machine and create/copy the files and folders onto a dockerfile - folders such as the ‘www’ which may contain images, custom style sheet etc may not load properly into the container and in a readable form for Docker.
  • The project directory for containerizing your app should look something similar to the figure shown below.
    • A file-folder containing the app should be in the directory and that folder (Boston_App in this example) should contain all the needed files and folders.
    • The dockerfile which will be created is also placed in this directory.
      Please note the nesting and hierarchy here as this will dictate the format of the Dockerfile as well as the directory from which the Docker commands are executed.

Figure 8 - Files and format for seamless containerization

Figure 8: Files for use in creating docker containers and scaling app

Dockerfile Details

• The base Docker image used to build upon is the rocker/shiny
• Instructions to create a directory within the container and to run it.
• Install any necessary R libraries.
• Copy the entire app folder (Boston_app in this example) into the newly created directory in the container.
• Specifications on expose port.
• The command line to run the application on host and port of choice which are stipulated in the command line.
  Note: Given the dynamism of one’s IP address - stipulating the host and port ensures consistency and there are no firewall impediments to loading the app.
  Also, the details in the command line helps keep the app file in its pristine state.

The Dockerfile created and shown below is saved in the directory as shown in Figure 8.

Figure 9 - Dockerfile for creating image & container for Shiny Application

Figure 9: Docker File For Boston Housing Market Shiny App

Your dockerfile is akin to a recipe that provides instructions to Docker to develop a customized container that meets your needs. When you run the image built from the dockerfile, an instance of a container is created - an isolated lightweight software environment customized for your application.

The following visual Figure 10 shows the docker commands executed on the terminal. The image is first built and subsequently the container is created from that image.

Figure 10 - Docker Commands for building image and container

Figure 10: Docker Commands for creating Image & Container from Dockerfile

Figure 11 below highlights the image and container of our Shiny application.

Figure 11 - Image & Container for scaling Shiny App

Figure 11: Image & Container for Shiny App

Note that adjacent to the container and image columns is the link to the port on which the app is hosted. When you click the port link when the container is “running” the shiny application is hosted on your localhost browser as shown below in Figure 12

Figure 12 - Boston Housing Market Shiny App launched from Docker

Figure 12: Shiny App launched from Docker

Images and containers created on your local machine are easily portable to your preferred cloud platform.
Additionally, the models and other files that are called into the Shiny application can be automated for recency and processed periodically in the cloud.

Conclusion

Glad you have made it to the end of this blog. Sharing one’s learning experience can be purposeful and a potent teacher. Hope this spurs you - the brilliant analyst who may be exploring or are indecisive about building Shiny or other user interactive applications;

• It can be a rewarding experience to take your analytic work to the next level.
• You have the power to develop value-added, user-friendly amazing applications.
• And for those who have amazing applications gathering dust on your local machine, wary and frustrated on scaling/ deploying them - you now have the confidence to take them to the next level.
• And for those that are nimble and versatile in building & scaling amazing customized applications, may your applications be a beacon of light!

References

. Posit Shiny Articles. https://shiny.posit.co/r/articles/.

———. Rocker Project Shiny. https://rocker-project.org/images/versioned/shiny.html.

———. Setup Guide Shinyapps.io. https://docs.posit.co/shinyapps.io/guide/getting_started/#creating-a-shinyapps.io-account.