bicountbrms 1.0.1
Documentation
- The examples for
posterior_predict_y1_and_y2()andposterior_predict_y1_given_y2()now fit the model withbf_mfd()and a prior on each coordinate, as invignette("prediction"). Under the brms default priors, each example took 50 to 60 minutes to fit on four cores, and 28% of transitions were divergent.
bicountbrms 1.0.0
Breaking changes
posterior_predict()andposterior_epred()return draws and the mean of the marginal distribution ofy1,mu + lambdaone, for fits from all four constructors. Up to 0.10.1,posterior_predict()returned draws ofy1from its distribution conditional on the recordedy2, andposterior_epred()returned the conditional expectation E[y_1 \mid y_2] = E[N_s \mid y_2] + \lambda_1, on every row, paired and unpaired alike. On an unpaired row, these conditional draws are imputations of the missingy1. Imputations are now returned byposterior_predict_y1_given_y2()andposterior_epred_y1_given_y2(), which return the same quantities as the oldposterior_predict()andposterior_epred()withkeep_observed = FALSE; given the same seed, the draws are identical.pp_check(),fitted(),conditional_effects(), andbayes_R2()therefore behave as they do for any brms family.log_lik(), and with itloo(),waic(), andkfold(), is unchanged.binegbin_mfd_to_dpars()andbinegbin_dpars_to_mfd()are renamedmfd_to_dpars()anddpars_to_mfd(), because both apply tobipoisas well as tobinegbin. The old names are removed.
New features
bf_mfd()builds the brms formula for a model in the congruence–bias coordinates: the midpointMof the two expected counts, the congruencef, the source bias throughdelta, and the overdispersionskappa. Each coordinate takes its own formula, as inbf_mfd(y1 | vint(y2), logM ~ 0 + site, logitf ~ 1, delta ~ 1). The set of coordinates supplied determines the dispersion structure, andkappax ~ 1gives the two exclusive components one overdispersion.rbicount()draws pairs of counts from the generating model ofbipoisandbinegbin, and returns the shared and source-exclusive components with the two counts. Its arguments are the names returned bymfd_to_dpars().posterior_predict_y1_given_y2()andposterior_epred_y1_given_y2()return draws and expectations ofy1conditional on the recordedy2. On an unpaired row, the draws impute the missingy1. Withkeep_observed = TRUE, the draws on paired rows are the recordedy1.posterior_predict_y1_and_y2()andposterior_epred_y1_and_y2()return draws and expectations of both counts, as an array of draws by rows by 2, with one draw of the shared component for each pair.
Other changes
The Stan functions
bipois_lpmfandbinegbin_lpmfreject an observation indicator other than 0 or 1, with a message that gives the ordervint(y2, y1_obs), and the R functions behindlog_lik()stop on the same condition. brms binds the arguments ofvint()by position, so a model written withvint(y1_obs, y2)previously compiled and sampled without error. The check does not detect the reversed order wheny2is 0 or 1 on every row.DESCRIPTION states
Depends: R (>= 4.1.0)andImports: brms (>= 2.21.0), and aSystemRequirementsfield names the C++17 compiler needed to fit a model.
Documentation
Getting started, The congruence–bias parameterisation, and Partially paired designs are rewritten around
bf_mfd()andrbicount(), and the new vignette Prediction and imputation describes the prediction functions and the imputation ofy1.The documentation follows the terminology of the manuscript describing the package: paired and unpaired rows, partially paired designs, source-exclusive components, dispersion and overdispersion, and the midpoint, congruence, and source bias.
The suggested value of
y1on an unpaired row is-1, because it cannot be confused with a recorded count.
bicountbrms 0.10.0
-
Breaking: four families become two, with two constructors each. The package now supplies
bipois()andbinegbin(), each with a_partialobs()sibling:fully paired first count missing on some rows Poisson bipois()bipois_partialobs()Negative-Binomial binegbin()binegbin_partialobs()Both constructors of a pair return the same
custom_familyname, so there is one Stan_lpmfand one set oflog_lik_*/posterior_predict_*/posterior_epred_*methods per component distribution. They differ only in whether the observation flag reaches the likelihood as data or as a constant. A user fitting fully paired data never has to supply the flag, or know it exists.bipois_cens(),binegbin_cens()and the fivebinegbin_jointnames are removed outright. There is no deprecation layer and nothing left to remove at a later version; see the last bullet for what to do if you hold a fit made under those names. -
binegbin()now has six dpars.shapexis gone; both excess components have their own dispersion,shapexoneandshapextwo, asbinegbin_cens()has had since 0.8.0. This is the change the release exists for. The capability was on the wrong family: in a partially paired design the unmatched rows inform onlyshapextwo, soshapexoneis identified by the matched rows alone, whereas in a fully paired design every row informs both and they are as easy to identify as they ever get.The symmetric model is a formula constraint, exactly as it has been for the partially observed family since 0.8.0:
bf(y1 | vint(y2) ~ 1, nlf(shapexone ~ shapexx), nlf(shapextwo ~ shapexx), shapexx ~ 1, ..., nl = TRUE)Set priors on it with
nlpar = "shapexx"rather thandpar = "shapex". _censnamed the wrong mechanism. Censoring means a value is known to lie in a set. On these rows, the first count is not observed at all and the likelihood marginalises over its whole support. brms already usescens()as an addition term meaning exactly the bounded thing —left,right,interval, with no code for “not observed” — so a brms user met a familiar word attached to an incompatible mechanism.partialobsnames what is actually partial: the pair, which is a property of the data.unobsandunmatchedname only the minority row class, andpartialalone collides with partial likelihood and partial pooling.-
One Stan function for both constructors, through a literal in
vars. brms builds the generated lpmf call by pasting each entry offamily$varsin verbatim, and validates those entries no further thanas.character(). The plain constructors exploit that: they declarevars = c("vint1[n]", "1"), so a fully paired model calls the same function withy1_obsfixed at the literal1.binegbin() target += binegbin_lpmf(Y[n] | ..., vint1[n], 1); binegbin_partialobs() target += binegbin_lpmf(Y[n] | ..., vint1[n], vint2[n]);The alternative — two Stan functions of the same name and different arity — needs user-defined function overloading, which arrived in Stan 2.29 (February 2022). Taking it would have obliged
DESCRIPTIONto declare floors on rstan and cmdstanr, whose absence would otherwise surface as an opaque compile error. This design needs none, and none has been added.That pasting behaviour is not a documented brms guarantee, so
tests/testthat/test-stancode-shape.Rpins it: it asserts the exact call each constructor generates and that the one-vintstandata()contains novint2. It needs brms but no Stan toolchain, so it runs in the fast suite. -
New tests for the fully paired path. Most users take the fully paired path and, before this release, nothing tested its prediction, expectation or dispersion routing: with a single
shapexthere was nothing to get wrong.-
tests/testthat/test-partialobs-predict.Rreplacestest-cens-predict.R. It keeps all five blocks from that file and adds a one-vintmirror of each: the same fixtures withshapexoneandshapextwoan order of magnitude apart, on a prep with novint2. -
tests/testthat/test-unified-vint.Rpins the release criterion that the two shapes agree: for each oflog_lik,posterior_predictandposterior_epred, the one-vintpath on matched data equals the two-vintpath with the flag set to 1.log_likis the only method that reads the flag, so its non-vacuity partner asserts the flag-0 answer differs; the other two ignore the flag by design, and that is asserted rather than left implicit. -
tests/testthat/test-dpar-compat.Rgains the stored-fit shape (below).
-
Stored
binegbinfits keep working. A fit stores its own family object, and brms resolves post-processing from the name it stored, so a five-dparbinegbinfit lands on the new six-dpar methods however this package is pinned. Three independent fallbacks have to line up for that to work:.get_rate()resolves pre-0.7.0lambdaem/lambdalb;.SHAPEXONE_NAMESand.SHAPEXTWO_NAMESboth listshapexlast, so both per-margin dispersions resolve to the singleshapexof such a fit; and an absentvint2selects the matched branch. Scanning the fit directory of the sibling project found 103 storedbinegbinfits, every one of them with the pre-0.7.0 rate names and the singleshapex, so all three paths are exercised in practice.test-dpar-compat.Rnow builds that exact prep and asserts all three methods agree with the six-dpar answer with the dispersions tied.-
binegbin_mfd_to_dpars()no longer emitsshapex. Supplyingkappaxnow writesshapexoneandshapextwoat that common value — the tied, symmetric model — rather than a singleshapex, which is no longer a dpar of anything and so could not be passed tobrm().kappaxsurvives as the shorthand it always was; only the output naming changed.binegbin_dpars_to_mfd()still acceptsshapex, because its input is a stored fit and that is the genuine dpar name on anybinegbinfit made before this release. The two directions differ deliberately: the forward one writes the current vocabulary, the inverse reads the old one, which is the same principle.get_dpar_any()applies. -
The README is now an orientation page. The construction, the moments, prior guidance, validation and the release history have moved into the articles. The README now contains a description of the package, installation instructions, a table of the constructors, a table mapping each symbol to its distributional-parameter name, one minimal fit, and links.
Four articles are new, and the site gains an
articles:section, which it did not have before: neither the get-started vignette norpaired-count-anatomywas previously placed on it deliberately.- The families and their parameters — the construction, the moments, the native distributional parameters, and a fitted demonstration that two excess dispersions an order of magnitude apart are resolved as different rather than merely bracketed.
-
Choosing priors — the improper-defaults fact, the recipes, and which of the
class,dparandnlparslots each prior belongs in. It routes on topaired-count-anatomyfor shrinkage priors in the interpretable coordinates. - A worked partially observed fit — the first such example in the package. A design with 40% of first counts unrecorded is simulated, fitted, and the imputed first count scored against the values withheld from the model.
- Migration and errata — what each release changed, the substitutions for existing code, and the two corrected numerical results.
The recommended priors in the (M, f, \delta) coordinates have changed. The anatomy of a paired count now recommends
normal(0, 0.5)onmethd, andnormal(0, 1)withlb = 0, a half-normal, on each ofkappasandkappax. The previous recommendation wasdouble_exponential(0, 0.5)andexponential(1). The worked fit in that article was refitted under the new priors: every 90% interval still covers its true value, and no posterior median moved by more than 1%.exponential(1)is documented as an alternative of the same shape with a heavier tail.Those priors are no longer called penalised-complexity priors. They follow the same principles: a base model at zero, and a density decreasing monotonically away from it. The PC construction additionally requires a distance measure derived for the specific model and a constant-rate assumption on that distance, neither of which is established for these families. The wording is corrected in
README.md, the get-started vignette, Choosing priors and The anatomy of a paired count.-
Attributions and citations updated. The negative-binomial construction is now credited to Kirkpatrick & Neale (2016, 2022), and the Poisson case to Holgate
- and Karlis & Ntzoufras (2003).
Precompiled vignette sources renamed.
<name>.Rmd.origis now_<name>.Rmd, so an editor highlights it as R Markdown; pkgdown skips any basename beginning with_. The sources do not ship, so this affects contributors only.-
Roxygen markdown is enabled, which changes every help page.
DESCRIPTIONgainedRoxygen: list(markdown = TRUE), which this package had never set, so roxygen had been passing markdown through to Rd uninterpreted. Backticks rendered as literal characters rather than as code formatting, cross-references between topics as bracketed text rather than as links, and emphasis markers as their own asterisks. Regenerating the six Rd files corrected all three.One line was restored rather than reformatted.
%opens a comment in Rd, so the unescaped%in%in the example that tells the two constructors apart truncated its own line at render time:On
?bipois_partialobs,?binegbin_partialobsand the corresponding reference pages of the site, that line ended after"vint2". It is now complete. Every documented function states what it returns. The
bipoisandbinegbintopics document five functions each and had a single\value{}describing the constructor alone. Both now give the return value and shape oflog_lik_*,posterior_predict_*andposterior_epred_*separately, as well as that of the constructor and its_stanvars()companion. The two_partialobstopics document two functions each, and state that the family returned shares itsname— and therefore its post-processing methods — with the fully paired constructor.Title:andDescription:also quote'brms', which is the convention CRAN applies to software names.If you hold a fit made with
bipois_cens(),binegbin_cens()orbinegbin_joint(), install the last version that defined them — 0.9.1 — and keep it on the search path for that fit. brms resolves post-processing late, by looking uplog_lik_<name>from the stored family name at call time, soloo()andposterior_predict()need the methods of the version the fit was made under. No argument to those calls can route around it, and 0.10.0 deliberately ships no shim: pinning the whole package is a stronger guarantee than a hand-picked subset of forwarders. # bicountbrms 0.9.1No behaviour change. Nothing in the likelihoods, predictions or expectations of the four families differs from 0.9.0. This release adds test coverage that was missing and corrects a claim in the README the suite did not support. No fitted model is affected and no code needs changing.
-
New tests: prediction on censored rows, with the two excess dispersions distinct (
tests/testthat/test-cens-predict.R). The_censfamilies exist to impute the first margin on rows where it was never observed, and that row class had no correctness test. The Monte Carlo check validatingposterior_predictagainst the exact conditionalP(y1 | y2)ran aty1_obs == 1in bothtest-binegbin_cens.Randtest-bipois_cens.R— matched rows, the one class where it cannot fail. Every prediction test also built its prep from a single five-dparshapex, which.SHAPEXONE_NAMESand.SHAPEXTWO_NAMESboth resolve to, so the two dispersions were never distinguished from each other.That combination left a specific defect invisible. In
posterior_predict_binegbin_cens(), the conditional split ofN_shared | y2is weighted byshapextwowhile the freshN1added on top takesshapexone; exchanging them passed 412 assertions across six test files with no failures. The new file fails on it four times. It also pins thaty1_obsdoes not change the draws — the flag selects a likelihood branch, not a prediction — and thaty2 = 0leavesposterior_predictdrawingNB2(lambdaone, shapexone)alone, which is the short-circuit keepingsample()from being handed an empty support. README: the conditional-prediction identity is now claimed accurately. The Testing section asserted it for the
_censfamilies without qualification while the suite checked it on matched rows only. The README now states that the identity holds on matched and censored rows whenshapexoneandshapextwodiffer. The tests in the preceding entry check the identity on matched rows and on censored rows, withshapexoneandshapextwoset to different values.
bicountbrms 0.9.0
-
The package has been split, and renamed.
pairedcountbrmscarried two unrelated suites: families that model a count pair jointly, and families that model its difference. They shared no code — not a helper, not a Stan block, not a dependency. This release separates them. The joint families arebicountbrms; the difference families (skellam1/skellam2,dnorm1/dnorm2,dlaplace1/dlaplace2) have returned toskellambrms, the name under which they were originally released.No fitted model needs refitting, either for the split or for the rename below. For each fit, brms resolves the
log_lik_*/posterior_predict_*/posterior_epred_*methods off the attached search path at call time, so a stored fit works as soon as the package holding its family is attached. No dpar name changed. The only source change the split itself requires islibrary(pairedcountbrms)→library(bicountbrms)orlibrary(skellambrms), and anypairedcountbrms::prefix.The GitHub repository is renamed to
anhsmith/bicountbrms, and GitHub serves a permanent redirect fromanhsmith/pairedcountbrmsfor both web and git, so existing clones andpak::pak("anhsmith/pairedcountbrms")keep working. The redirect fromanhsmith/skellambrmsno longer applies: that name is now an independent repository holding the difference families. A user who installsskellambrmstherefore receives the difference families, not this package.The timing is deliberate. A forthcoming 1.0.0 will be archived and assigned a DOI, and cited from a methods article describing the joint families; the citable artefact should therefore describe one thing rather than half of two. This 0.9.0 makes the split and the additions below available for checking first, so that what is archived has been exercised rather than merely tested.
-
Renamed:
binegbin_joint()→binegbin_cens(). All four families in this package model the two counts jointly, so_jointnamed a property common to all of them. The property that selects this one is that the first margin may be unobserved on some rows, which_censnames.binegbin_cens_stanvars(),binegbin_cens_lpmf(Stan) and the three post-processing methods follow the same rename.Old code and old fits both keep working.
binegbin_joint()andbinegbin_joint_stanvars()remain as deprecated wrappers that warn once per call and return the new objects. The three post-processing functionslog_lik_binegbin_joint(),posterior_predict_binegbin_joint()andposterior_epred_binegbin_joint()remain as silent forwarders, because a fit made before this release storesfamily$name == "binegbin_joint"permanently and brms builds its method names from that stored name — nothing the caller passes toloo()orposterior_predict()can route around it. All five are removed in the next major version.No dpar name changed, so the forwarding is a straight hand-off, and a five-dpar pre-0.8.0 fit still resolves through both compatibility layers at once (a test pins this).
-
New family:
bipois_cens()/bipois_cens_stanvars(). The censoring-aware bivariate Poisson — the equidispersed counterpart ofbinegbin_cens(), and the censoring-aware counterpart ofbipois(). Three log-linked dpars (mu,lambdaone,lambdatwo) and twovint()integers,vint(y2, y1_obs), exactly asbinegbin_cens()takes them. It has no deprecated aliases: it was added in this release under its final name.Its censored branch is closed form. A sum of independent Poissons is Poisson, so the
y1-integrated marginal collapses topoisson_lpmf(y2 | mu + lambdatwo)— no convolution sum, no cutoff.binegbin_cens()cannot do this (NB2 + NB2 is not NB2 unless the two components share a success probability), which is why it evaluates that branch term by term.Two things follow. Users whose counts are equidispersed no longer have to fit
binegbin_cens()with its dispersions pressed against the Poisson boundary, where sampling degrades. And the suite gains an independent analytic reference for the censored branch ofbinegbin_cens(): driving its three dispersions to their Poisson limit must reproducebipois_cens(), checked as a limit — the error must shrink in proportion to1/phi— rather than at a single tolerance. Previously that branch was pinned only by the marginal identity, which compares it against the matched branch of the same code and so shares any error common to both sums. -
Breaking (numerical):
posterior_epred_binegbin()is now exact. It previously substituted the marginal shared fractionmu/(mu + lambdatwo)for the conditional one — thebipois()answer, which is exact only in the Poisson limit — and its docstring described this as a point approximation adequate for display. It is not adequate. Over a grid of plausible rates and dispersions, withy2within one standard deviation of its mean, the substitution was in error by more than 5% in about half the settings and more than 20% in a third, in either direction depending on which component carried the greater dispersion.E[N_shared | y2]is now evaluated as the mean of the discrete conditional overk = 0..y2— the same weightsposterior_predict_binegbin()already samples from, summed rather than sampled — so the expectation and the predictions agree by construction rather than approximately.Numbers produced by
posterior_epred_binegbin()in an earlier version should be recomputed.log_lik(),loo()andposterior_predict()are unaffected; the likelihood has not changed. New:
posterior_epred_binegbin_cens(). That family previously defined noposterior_epredat all, on the view thatE[y1]is ambiguous wheny1is unobserved. It is not:E[y1 | y2]is well defined whether or noty1was recorded, and it is the quantity a user imputing the missing margin wants. It is returned for every row, matched and censored alike, matching the existing convention ofposterior_predict_binegbin_cens(). Stored fits reach it through the deprecatedposterior_epred_binegbin_joint()forwarder, so they gain the method without refitting.All four families now share one
posterior_epredconvention, stated once intests/testthat/test-epred.Rand held to one standard: the expectation must equal the mean ofposterior_predictdraws from the same family to within Monte Carlo error. Before this release, the three families answered three different ways — exactly, approximately, and not at all.skellammoves from Imports to Suggests. No function inR/calls it now that the Skellam families have left;tests/testthat/test-bipois.Rstill usesskellam::dskellam()as the independent reference for the induced difference distribution, which is worth keeping.The environment variable for the coverage gate is renamed
PAIREDCOUNTBRMS_COVERAGE→BICOUNTBRMS_COVERAGE.The README and the getting-started vignette are rewritten around the four joint families, and the vignette section “one limitation to know about” — which described how
posterior_epred()in brms fails on truncated custom-family fits — is replaced by a statement of the epred convention above. That limitation cannot arise here: these families do not supportresp_trunc(). It applies to the difference families and is now documented inskellambrms.
Entries below record the history of this code under its former names. It was skellambrms through 0.5.0 and pairedcountbrms from 0.6.0 to 0.8.0, and those packages contained the difference families as well, so some entries describe families that are no longer here. They are left as written rather than retrospectively edited. Releases before 0.4.0, which predate the joint families entirely, are recorded in the skellambrms NEWS.