Modern Approaches to Quality in Open-Source Software

R/Pharma Summit (9-15-2025)

Eric Nantz (Eli Lilly)

Paulo Bargo (Novartis)

Harvey Liberman (Novartis)

slido.com

#9162726

Code/Software Spectrum

Statistical Programs

• TLF generation

Multi-use Tools

• Shiny App
• R/Python packages

Systems

• Clinical data repository
• Commercial Software

• Might be multi-use, but not be “multi-year”
• Other factors are important, such as people maintaining, complexity
• Another dimension: What is developed internally versus externally (pharmaverse vs internal package)
• What constitutes an overhaul that requires a re-assessment of risk and validation?
• We will never get an industry consensus on this, but it would be great to have at least some of the concepts are shared across the companies. The white paper can serve as a reference
• Also if we develop an internal capability that is open-sourced, how do we handle validation
• What constitutes an “IT System”
• Focus on the capability itself, which likely integrates with already-validated systems
  • Hosting platforms (Posit Connect)
  • Authentication providers (SSO)
  • HPC clusters
  • Cloud services (AWS, Azure)

One-Size Fits All?

• Hard to address all software code efficiently in one SOP
• Development vs. usage of a re-usable tool

Sonic Sonic The Hedgehog GIFfrom Sonic GIFs

Foundational Principles

Documentation

User-facing

• Package documentation using roxygen2
• Vignettes
• Website with pkgdown
• Clear links to code repository

Quality

• Specifications / Requirements
• Key contacts (author, maintainer)
• Test results
• Issue board
• Audit trails

Version Control with Git

The standard in modern software development

• Branching, tagging are critical for collaborative development
• Platforms: GitHub, GitLab, and more
• Learning curve can be steep, but future you will thank you!

Testing

• testthat used across spectrum of R packages and Shiny applications
• Emerging paradigms: Behavior-Driven and Test-Driven Development
• GitHub Actions and GitLab Runners provide automation to execute tests in a code repository

Additional Principles

• Release Management
• Maintenance
• Retirement

Intangibles

• Investment in upskilling quality partners on software dev best practices
  • Power of demonstrations
• Eliminating resistance to change because “it was not done before”
• Embracing new approaches (not falling back to the same principles which may not apply)
• Knowledge Management / Turnover

• Proof of concepts are a major driving force of success to implement new changes
• We should have a sandbox repo related to documentation, another for testing, another for failing a test. Imagine a posit cloud being available to a user to try this out on a repository.
• Discussion point: acceptance of validation approach for a reviewer (social collatoral). The are accountable for their decision, often times they will side with the most conservative approach.
• Imagine that the treatment your code is informing where the patient is a family member

Quality Whitepaper

• Golden opportunity to document these ideas as a key reference
• Inspired by success of R Validaion Hub Whitepaper
• Your contributions are welcome!

Image credit: https://www.deviantart.com/doa687/art/Sonic-Mania-Animation-Redux-Green-Hill-Zone-749688233