The data-informed Parametric Bootstrap Cross-fitting Method (PBCM) generates synthetic data from two models of a phenomenon parameterized by fits to an empirical dataset, and then cross-fits the models to these data. The result is two distributions of the goodness of fit metric \(\Delta GoF = GoF_1 - GoF_2\), where \(GoF_1\) is the fit of model 1 and \(GoF_2\) the fit of model 2.
pbcm.di( data, fun1, fun2, genfun1, genfun2, reps, args1 = NULL, args2 = NULL, genargs1 = NULL, genargs2 = NULL, print_genargs = TRUE, nonparametric_bootstrap = TRUE, verbose = TRUE, GoFname = "GoF" )
| data | Data frame |
|---|---|
| fun1 | First modelling function |
| fun2 | Second modelling function |
| genfun1 | Generator function for first model |
| genfun2 | Generator function for second model |
| reps | Number of Monte Carlo repetitions |
| args1 | List of arguments passed to |
| args2 | List of arguments passed to |
| genargs1 | List of arguments passed to |
| genargs2 | List of arguments passed to |
| print_genargs | Whether the generator argument values should be included in output (see Details) |
| nonparametric_bootstrap | Whether |
| verbose | If |
| GoFname | Name of the element returned by |
A data frame in long format with the following columns:
repMonte Carlo repetition number
generatorGenerating model
GoF1Goodness of fit of model 1
GoF2Goodness of fit of model 2
DeltaGoFEquals GoF1 - GoF2
In addition to these columns, if print_genargs == TRUE, each argument in the lists genargs1 and genargs2 is included as a column of its own, with the argument's name prefixed by "genargs1_" or "genargs2_".
Functions fun1 and fun2 must take data as an argument in addition to any arguments specified in args1 and args2. Moreover, these functions must return a list with at least one element carrying the goodness of fit; the name of this element may be specified through the GoFname argument, by default the string "GoF" is assumed. Functions genfun1 and genfun2 must take an argument named model (the output of fun1 and fun2).
Wagenmakers, E.-J., Ratcliff, R., Gomez, P. & Iverson, G. J. (2004) Assessing model mimicry using the parametric bootstrap. Journal of Mathematical Psychology, 48(1), 28–50. https://doi.org/10.1016/j.jmp.2003.11.004
x <- seq(from=0, to=1, length.out=100) mockdata <- data.frame(x=x, y=x + rnorm(100, 0, 0.5)) myfitfun <- function(data, p) { res <- nls(y~a*x^p, data, start=list(a=1.1)) list(a=coef(res), GoF=deviance(res)) } mygenfun <- function(model, p) { x <- seq(from=0, to=1, length.out=100) y <- model$a*x^p + rnorm(100, 0, 0.5) data.frame(x=x, y=y) } pbcm.di(data=mockdata, fun1=myfitfun, fun2=myfitfun, genfun1=mygenfun, genfun2=mygenfun, reps=20, args1=list(p=1), args2=list(p=2), genargs1=list(p=1), genargs2=list(p=2))#> Initializing output data frame... #> Bootstrapping... #> | | | 0% | |==== | 5% | |======= | 10% | |========== | 15% | |============== | 20% | |================== | 25% | |===================== | 30% | |======================== | 35% | |============================ | 40% | |================================ | 45% | |=================================== | 50% | |====================================== | 55% | |========================================== | 60% | |============================================== | 65% | |================================================= | 70% | |==================================================== | 75% | |======================================================== | 80% | |============================================================ | 85% | |=============================================================== | 90% | |================================================================== | 95% | |======================================================================| 100%#> genargs1_p genargs2_p rep generator GoF1 GoF2 DeltaGoF #> 1 1 NA 1 model1 26.59169 29.69149 -3.09979606 #> 2 1 NA 2 model1 20.11336 22.55904 -2.44567575 #> 3 1 NA 3 model1 23.94691 23.38980 0.55710570 #> 4 1 NA 4 model1 22.87248 23.29989 -0.42741297 #> 5 1 NA 5 model1 28.24804 28.50483 -0.25678283 #> 6 1 NA 6 model1 26.94672 30.66978 -3.72306499 #> 7 1 NA 7 model1 27.06818 26.94283 0.12535599 #> 8 1 NA 8 model1 30.33206 30.72636 -0.39429792 #> 9 1 NA 9 model1 30.91686 33.96364 -3.04677958 #> 10 1 NA 10 model1 22.58249 23.31667 -0.73417980 #> 11 1 NA 11 model1 27.63299 30.22351 -2.59051887 #> 12 1 NA 12 model1 26.24263 27.02143 -0.77880087 #> 13 1 NA 13 model1 30.93412 32.17314 -1.23902927 #> 14 1 NA 14 model1 26.22216 28.79895 -2.57678641 #> 15 1 NA 15 model1 26.97977 28.96358 -1.98381020 #> 16 1 NA 16 model1 22.94223 24.59594 -1.65371111 #> 17 1 NA 17 model1 20.40461 20.53054 -0.12593654 #> 18 1 NA 18 model1 27.33983 29.80956 -2.46973068 #> 19 1 NA 19 model1 26.91499 26.31365 0.60134407 #> 20 1 NA 20 model1 20.29942 20.49055 -0.19112761 #> 21 NA 2 1 model2 33.72142 30.97344 2.74798444 #> 22 NA 2 2 model2 27.61873 26.90102 0.71770452 #> 23 NA 2 3 model2 25.96907 25.62547 0.34360885 #> 24 NA 2 4 model2 23.62990 23.53024 0.09966179 #> 25 NA 2 5 model2 24.38894 24.21573 0.17320785 #> 26 NA 2 6 model2 24.98114 22.02453 2.95660969 #> 27 NA 2 7 model2 24.69730 22.34081 2.35649428 #> 28 NA 2 8 model2 20.96703 20.45273 0.51429577 #> 29 NA 2 9 model2 28.11880 27.38079 0.73800918 #> 30 NA 2 10 model2 23.39494 24.64437 -1.24943704 #> 31 NA 2 11 model2 22.49582 20.40660 2.08921604 #> 32 NA 2 12 model2 25.49898 24.11380 1.38517727 #> 33 NA 2 13 model2 27.51811 26.49302 1.02508829 #> 34 NA 2 14 model2 16.28560 14.39185 1.89375241 #> 35 NA 2 15 model2 22.54143 22.49313 0.04830329 #> 36 NA 2 16 model2 21.70470 20.59196 1.11274271 #> 37 NA 2 17 model2 28.90371 26.53741 2.36630960 #> 38 NA 2 18 model2 26.71399 24.44377 2.27022444 #> 39 NA 2 19 model2 27.10381 25.12799 1.97581952 #> 40 NA 2 20 model2 27.37560 25.31020 2.06540252