MCMC diagnostics
Run number 1100
To produce a set of MCMC diagnostic plots that will be included in a report. Please note that these plots are just meant to provide an example of what could be created and how. They are not an exhaustive list of every possible plot and were chosen with the project aims in mind.
Set up
Model location
Define modelName and path to the model directory
(MODEL_DIR).
Figure location
If saving figures out to pdf, define where those pdfs should be saved
to. Here the figures are saved to
deliv > figure > model_run_number
Model details
A summary of high-level model details.
Read in the model details using read_model. Details
stored in the mod object can be used to identify the
location of the source data (used in $DATA) - to see how this is done
look at the bbr::get_data_path() and
bbr::build_path_from_model() helper functions.
── Status ─────────────────────────────────
• Finished Running
── Absolute Model Path ───────────────────────────
• /data/bbr-nonmem-poppk-bayes/model/pk/1100
── YAML & Model Files ───────────────────────────
• /data/bbr-nonmem-poppk-bayes/model/pk/1100.yaml
• /data/bbr-nonmem-poppk-bayes/model/pk/1100.ctl
── Description ───────────────────────────────
• Final Model
── Tags ──────────────────────────────────
• two-compartment + absorption
• ETA-CL
• ETA-KA
• ETA-V2
• CLWT-allo
• V2WT-allo
• QWT-allo
• V3WT-allo
• proportional RUV
• CLEGFR
• CLAGE
• CLALB
── Notes ──────────────────────────────────
• 1: All diagnostics look reasonable
Read output
Parameter estimates
Compute summaries of posterior distributions as well as some diagnostics.
Bulk effective sample size (ESS) is a measure of sampling efficiency for the location of the distribution, while Tail ESS is a measure of sampling efficiency for the tails (5% and 95% quantiles) of the distribution. Higher values indicate greater sampling efficiency. A very rough rule of thumb is to aim for at least 400 for each parameter.
R-hat is a convergence diagnostic that compares the between- and within-chain variances of model parameters. Values close to 1 indicate that the chains have converged to similar distributions. Aim for less than about 1.05 for all parameters.
| parameter | mean | median | sd | mad | 90% CI | ess_bulk | ess_tail | rhat |
|---|---|---|---|---|---|---|---|---|
| THETA[1] | 0.478 | 0.478 | 0.0628 | 0.0577 | (0.373, 0.579) | 1.08e+03 | 1.3e+03 | 1 |
| THETA[2] | 4.1 | 4.1 | 0.0278 | 0.0279 | (4.06, 4.15) | 803 | 1.08e+03 | 1 |
| THETA[3] | 1.16 | 1.16 | 0.0291 | 0.029 | (1.11, 1.21) | 948 | 1.24e+03 | 1 |
| THETA[4] | 4.23 | 4.23 | 0.0248 | 0.0261 | (4.19, 4.27) | 1.83e+03 | 1.6e+03 | 1 |
| THETA[5] | 1.29 | 1.29 | 0.0379 | 0.0373 | (1.23, 1.35) | 2.12e+03 | 1.68e+03 | 1 |
| THETA[6] | 0.488 | 0.487 | 0.0485 | 0.0491 | (0.409, 0.568) | 1.38e+03 | 1.37e+03 | 1 |
| THETA[7] | -0.0413 | -0.0399 | 0.0736 | 0.0758 | (-0.163, 0.0766) | 1.1e+03 | 1.37e+03 | 1 |
| THETA[8] | 0.423 | 0.423 | 0.0853 | 0.086 | (0.282, 0.566) | 1.81e+03 | 1.6e+03 | 1 |
| OMEGA[1,1] | 0.237 | 0.231 | 0.0565 | 0.0536 | (0.155, 0.34) | 864 | 1.36e+03 | 1 |
| OMEGA[2,1] | 0.0658 | 0.0644 | 0.0199 | 0.0195 | (0.0359, 0.101) | 397 | 895 | 1.01 |
| OMEGA[2,2] | 0.0837 | 0.0831 | 0.0115 | 0.0114 | (0.0666, 0.104) | 594 | 943 | 1.01 |
| OMEGA[3,1] | 0.118 | 0.116 | 0.0225 | 0.0216 | (0.0852, 0.156) | 492 | 997 | 1 |
| OMEGA[3,2] | 0.0686 | 0.0679 | 0.0102 | 0.00969 | (0.0528, 0.0865) | 664 | 1.21e+03 | 1 |
| OMEGA[3,3] | 0.116 | 0.114 | 0.0136 | 0.0129 | (0.0949, 0.14) | 645 | 1.29e+03 | 1 |
| OMEGA[4,1] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[4,2] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[4,3] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[4,4] | 0.025 | 0.025 | 0 | 0 | (0.025, 0.025) | NA | NA | NA |
| OMEGA[5,1] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[5,2] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[5,3] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[5,4] | 0 | 0 | 0 | 0 | ( 0, 0) | NA | NA | NA |
| OMEGA[5,5] | 0.025 | 0.025 | 0 | 0 | (0.025, 0.025) | NA | NA | NA |
| SIGMA[1,1] | 0.0394 | 0.0394 | 0.00113 | 0.00116 | (0.0376, 0.0412) | 2.39e+03 | 1.5e+03 | 1 |
MCMC diagnostic plots
These plots assess whether all of the chains have converged to a single, stationary distribution.
Rhat
Pairs
Look for any strong correlation between parameters, indicating possible over-parameterisation.
ESS
ESS vs sample
Plots Bulk and Tail ESS versus iteration to check that ESS increases linearly.
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## [[6]]
##
## [[7]]
##
## [[8]]
##
## [[9]]
##
## [[10]]
##
## [[11]]
##
## [[12]]
##
## [[13]]
##
## [[14]]
##
## [[15]]
ESS vs quantile
Plots Bulk and Tail ESS versus quantile to better diagnose areas of the distributions that the iterative algorithm fails to explore efficiently.
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## [[6]]
##
## [[7]]
##
## [[8]]
##
## [[9]]
##
## [[10]]
##
## [[11]]
##
## [[12]]
##
## [[13]]
##
## [[14]]
##
## [[15]]
History plots
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
Rank plots
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
Density plots
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
Session details
It is considered good practice to include these details at the end of all rmd scripts
Sys.getenv("AMI_NAME")## [1] ""sessioninfo::session_info()## ─ Session info ───────────────────────────────────────────────────────────────────────────────────────────
## setting value
## version R version 4.1.3 (2022-03-10)
## os Ubuntu 18.04.6 LTS
## system x86_64, linux-gnu
## ui RStudio
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/New_York
## date 2024-05-12
## rstudio 2022.02.4+500.pro1 Prairie Trillium (server)
## pandoc 2.17.1.1 @ /usr/lib/rstudio-server/bin/quarto/bin/ (via rmarkdown)
##
## ─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────
## package * version date (UTC) lib source
## abind 1.4-5 2016-07-21 [1] CRAN (R 4.1.3)
## assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.1.3)
## backports 1.4.1 2021-12-13 [1] CRAN (R 4.1.3)
## bayesplot * 1.11.1 2024-02-15 [1] CRAN (R 4.1.3)
## bbr * 1.10.0 2024-03-07 [1] MPNDEV (R 4.1.3)
## bbr.bayes * 0.2.0 2024-03-07 [1] MPNDEV (R 4.1.3)
## bit 4.0.5 2022-11-15 [1] CRAN (R 4.1.3)
## bit64 4.0.5 2020-08-30 [1] CRAN (R 4.1.3)
## bslib 0.6.1 2023-11-28 [1] CRAN (R 4.1.3)
## cachem 1.0.8 2023-05-01 [1] CRAN (R 4.1.3)
## checkmate 2.3.1 2023-12-04 [1] CRAN (R 4.1.3)
## cli 3.6.2 2023-12-11 [1] CRAN (R 4.1.3)
## codetools 0.2-19 2023-02-01 [1] CRAN (R 4.1.3)
## colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.1.3)
## cowplot * 1.1.3 2024-01-22 [1] CRAN (R 4.1.3)
## crayon 1.5.2 2022-09-29 [1] CRAN (R 4.1.3)
## data.table 1.15.2 2024-02-29 [1] CRAN (R 4.1.3)
## diffobj 0.3.5 2021-10-05 [1] CRAN (R 4.1.3)
## digest 0.6.34 2024-01-11 [1] CRAN (R 4.1.3)
## distributional 0.4.0 2024-02-07 [1] CRAN (R 4.1.3)
## dplyr * 1.1.4 2023-11-17 [1] CRAN (R 4.1.3)
## evaluate 0.23 2023-11-01 [1] CRAN (R 4.1.3)
## fansi 1.0.6 2023-12-08 [1] CRAN (R 4.1.3)
## farver 2.1.1 2022-07-06 [1] CRAN (R 4.1.3)
## fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.1.3)
## forcats * 1.0.0 2023-01-29 [1] CRAN (R 4.1.3)
## fs 1.6.3 2023-07-20 [1] CRAN (R 4.1.3)
## furrr * 0.3.1 2022-08-15 [1] CRAN (R 4.1.3)
## future * 1.33.1 2023-12-22 [1] CRAN (R 4.1.3)
## future.apply 1.11.1 2023-12-21 [1] CRAN (R 4.1.3)
## generics 0.1.3 2022-07-05 [1] CRAN (R 4.1.3)
## GGally 2.2.1 2024-02-14 [1] CRAN (R 4.1.3)
## ggplot2 * 3.4.4 2023-10-12 [1] CRAN (R 4.1.3)
## ggstats 0.5.1 2023-11-21 [1] CRAN (R 4.1.3)
## globals 0.16.2 2022-11-21 [1] CRAN (R 4.1.3)
## glue * 1.7.0 2024-01-09 [1] CRAN (R 4.1.3)
## gridExtra 2.3 2017-09-09 [1] CRAN (R 4.1.3)
## gridGraphics 0.5-1 2020-12-13 [1] CRAN (R 4.1.3)
## gtable 0.3.4 2023-08-21 [1] CRAN (R 4.1.3)
## here * 1.0.1 2020-12-13 [1] CRAN (R 4.1.3)
## highr 0.10 2022-12-22 [1] CRAN (R 4.1.3)
## hms 1.1.3 2023-03-21 [1] CRAN (R 4.1.3)
## htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.1.3)
## inline 0.3.19 2021-05-31 [1] CRAN (R 4.1.3)
## jquerylib 0.1.4 2021-04-26 [1] CRAN (R 4.1.3)
## jsonlite 1.8.8 2023-12-04 [1] CRAN (R 4.1.3)
## knitr 1.45 2023-10-30 [1] CRAN (R 4.1.3)
## labeling 0.4.3 2023-08-29 [1] CRAN (R 4.1.3)
## lattice 0.22-5 2023-10-24 [1] CRAN (R 4.1.3)
## lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.1.3)
## listenv 0.9.1 2024-01-29 [1] CRAN (R 4.1.3)
## loo * 2.7.0 2024-02-24 [1] CRAN (R 4.1.3)
## lubridate * 1.9.3 2023-09-27 [1] CRAN (R 4.1.3)
## magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.1.3)
## Matrix 1.6-5 2024-01-11 [1] CRAN (R 4.1.3)
## matrixStats 1.2.0 2023-12-11 [1] CRAN (R 4.1.3)
## mclust 6.1 2024-02-23 [1] CRAN (R 4.1.3)
## mgcv 1.9-1 2023-12-21 [1] CRAN (R 4.1.3)
## mrggsave * 0.4.5 2023-10-17 [1] MPNDEV (R 4.1.3)
## mrgsolve * 1.4.1 2024-03-07 [1] MPNDEV (R 4.1.3)
## munsell 0.5.0 2018-06-12 [1] CRAN (R 4.1.3)
## nlme 3.1-164 2023-11-27 [1] CRAN (R 4.1.3)
## nmrec 0.4.0 2024-03-07 [1] MPNDEV (R 4.1.3)
## npde 3.5 2024-02-07 [1] CRAN (R 4.1.3)
## parallelly 1.37.1 2024-02-29 [1] CRAN (R 4.1.3)
## patchwork * 1.2.0 2024-01-08 [1] CRAN (R 4.1.3)
## pillar 1.9.0 2023-03-22 [1] CRAN (R 4.1.3)
## pkgbuild 1.4.3 2023-12-10 [1] CRAN (R 4.1.3)
## pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.3)
## plyr 1.8.9 2023-10-02 [1] CRAN (R 4.1.3)
## pmplots * 0.4.0 2024-03-07 [1] MPNDEV (R 4.1.3)
## pmtables * 0.6.0 2023-10-17 [1] MPNDEV (R 4.1.3)
## posterior * 1.5.0 2023-10-31 [1] CRAN (R 4.1.3)
## prettyunits 1.2.0 2023-09-24 [1] CRAN (R 4.1.3)
## processx 3.8.3 2023-12-10 [1] CRAN (R 4.1.3)
## progress 1.2.3 2023-12-06 [1] CRAN (R 4.1.3)
## progressr 0.14.0 2023-08-10 [1] CRAN (R 4.1.3)
## ps 1.7.6 2024-01-18 [1] CRAN (R 4.1.3)
## purrr * 1.0.2 2023-08-10 [1] CRAN (R 4.1.3)
## QuickJSR 1.1.3 2024-01-31 [1] CRAN (R 4.1.3)
## R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.3)
## RColorBrewer 1.1-3 2022-04-03 [1] CRAN (R 4.1.3)
## Rcpp 1.0.12 2024-01-09 [1] CRAN (R 4.1.3)
## RcppParallel 5.1.7 2023-02-27 [1] CRAN (R 4.1.3)
## readr * 2.1.5 2024-01-10 [1] CRAN (R 4.1.3)
## renv 0.16.0 2022-09-29 [1] CRAN (R 4.1.3)
## reshape2 1.4.4 2020-04-09 [1] CRAN (R 4.1.3)
## rlang 1.1.3 2024-01-10 [1] CRAN (R 4.1.3)
## rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.1.3)
## rprojroot 2.0.4 2023-11-05 [1] CRAN (R 4.1.3)
## rstan 2.32.5 2024-01-10 [1] CRAN (R 4.1.3)
## rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.1.3)
## sass 0.4.8 2023-12-06 [1] CRAN (R 4.1.3)
## scales 1.3.0 2023-11-28 [1] CRAN (R 4.1.3)
## sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.1.3)
## StanHeaders 2.32.6 2024-03-01 [1] CRAN (R 4.1.3)
## stringi 1.8.3 2023-12-11 [1] CRAN (R 4.1.3)
## stringr * 1.5.1 2023-11-14 [1] CRAN (R 4.1.3)
## tensorA 0.36.2.1 2023-12-13 [1] CRAN (R 4.1.3)
## tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.1.3)
## tidyr * 1.3.1 2024-01-24 [1] CRAN (R 4.1.3)
## tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.1.3)
## tidyverse * 2.0.0 2023-02-22 [1] CRAN (R 4.1.3)
## timechange 0.3.0 2024-01-18 [1] CRAN (R 4.1.3)
## tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.1.3)
## utf8 1.2.4 2023-10-22 [1] CRAN (R 4.1.3)
## vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.1.3)
## vroom 1.6.5 2023-12-05 [1] CRAN (R 4.1.3)
## withr 3.0.0 2024-01-16 [1] CRAN (R 4.1.3)
## xfun 0.42 2024-02-08 [1] CRAN (R 4.1.3)
## xtable 1.8-4 2019-04-21 [1] CRAN (R 4.1.3)
## yaml 2.3.8 2023-12-11 [1] CRAN (R 4.1.3)
## yspec * 0.6.1 2023-10-17 [1] MPNDEV (R 4.1.3)
##
## [1] /data/bbr-nonmem-poppk-bayes/renv/library/R-4.1/x86_64-pc-linux-gnu
## [2] /data/home/timw/R/renvExt
## [3] /data/bbr-nonmem-poppk-bayes/renv/sandbox/R-4.1/x86_64-pc-linux-gnu/dbd573bd
##
## ──────────────────────────────────────────────────────────────────────────────────────────────────────────bbr::bbi_version()## [1] "3.3.0"