Base classes and utility functions

Base classes

Base Bayes model and A/B testing classes.

class cprior.cdist.base.BayesABTest(modelA, modelB, simulations=None, random_state=None)

Bases: object

Bayes A/B test abstract class.

Parameters:
  • modelA (object) – The Bayes model for variant A.
  • modelB (object) – The Bayes model for variant B.
  • simulations (int or None (default=1000000)) – Number of Monte Carlo simulations.
  • random_state (int or None (default=None)) – The seed used by the random number generator.
expected_loss()

Compute the expected loss. This is the expected uplift lost by choosing a given variant.

expected_loss_ci()

Compute credible intervals on the difference distribution of \(Z = B-A\) and/or \(Z = A-B\).

expected_loss_relative()

Compute expected relative loss for choosing a variant. This can be seen as the negative expected relative improvement or uplift.

expected_loss_relative_ci()

Compute credible intervals on the relative difference distribution of \(Z = (B-A)/A\) and/or \(Z = (A-B)/B\).

probability()

Compute the error probability or chance to beat control.

update_A(data)

Update posterior parameters for variant A with new data samples.

Parameters:data (array-like, shape = (n_samples)) –
update_B(data)

Update posterior parameters for variant B with new data samples.

Parameters:data (array-like, shape = (n_samples)) –
class cprior.cdist.base.BayesMVTest(models, simulations=None, random_state=None, n_jobs=None)

Bases: object

Bayesian Multivariate test abstract class.

Parameters:
  • models (dict) – The Bayes models.
  • simulations (int or None (default=1000000)) – Number of Monte Carlo simulations.
  • random_state (int or None (default=None)) – The seed used by the random number generator.
expected_loss()

Compute the expected loss. This is the expected uplift lost by choosing a given variant.

expected_loss_ci()

Compute credible intervals on the difference between two variants.

expected_loss_relative()

Compute expected relative loss for choosing a variant. This can be seen as the negative expected relative improvement or uplift.

expected_loss_relative_ci()

Compute credible intervals on the relative difference between two variants.

expected_loss_vs_all()

Compute the expected loss against all variations.

probability()

Compute the error probability or chance to beat control.

probability_vs_all()

Compute the error probability or chance to beat all variations.

update(data, variant)

Update posterior parameters for a given variant with new data samples.

Parameters:
  • data (array-like, shape = (n_samples)) –
  • variant (str) –
class cprior.cdist.base.BayesModel(name='')

Bases: object

Bayes model class.

Parameters:name (str (default="")) – Model name.
cdf(x)

Cumulative distribution function of the posterior distribution.

Parameters:x (array-like) – Quantiles.
Returns:cdf – Cumulative distribution function evaluated at x.
Return type:numpy.ndarray
credible_interval(interval_length)

Credible interval of the posterior distribution.

Parameters:interval_length (float (default=0.9)) – Compute interval_length% credible interval. This is a value in [0, 1].
Returns:interval – Lower and upper credible interval limits.
Return type:tuple
mean()

Mean of the posterior distribution.

pdf(x)

Probability density function of the posterior distribution.

Parameters:x (array-like) – Quantiles.
Returns:pdf – Probability density function evaluated at x.
Return type:numpy.ndarray
ppf(q)

Percent point function (quantile) of the posterior distribution.

Parameters:x (array-like) – Lower tail probability.
Returns:ppf – Quantile corresponding to the lower tail probability q.
Return type:numpy.ndarray
rvs(size=1, random_state=None)

Random variates of the posterior distribution.

Parameters:
  • size (int (default=1)) – Number of random variates.
  • random_state (int or None (default=None)) – The seed used by the random number generator.
Returns:

rvs – Random variates of given size.
Return type:

numpy.ndarray or scalar
std()

Standard deviation of the posterior distribution.

var()

Variance of the posterior distribution.

Utility functions

Utilities to check methods and models.

cprior.cdist.utils.check_ab_method(method, method_options, variant, lift=0, interval_length=0.9)

Check parameters of A/B testing method.

Parameters:
  • method (str) – The default computational method.
  • method_options (list or tuple) – The list of supported computational methods.
  • variant (str) – The chosen variant. Options are “A”, “B”, “all”
  • lift (float (default=0.0)) – The amount of uplift.
cprior.cdist.utils.check_models(refclass, *models)

Check that models for A/B and multivariate testing belong to the correct class.

Parameters:
  • refclass (object) – Reference class.
  • models (objects) – Model instances to be checked.
cprior.cdist.utils.check_mv_method(method, method_options, control, variant, variants, lift=0, interval_length=0.9)

Check parameters of Multivariate testing method.

Parameters:
  • method (str) – The default computational method.
  • method_options (list or tuple) – The list of supported computational methods.
  • control (str) – The control variant.
  • variant (str) – The variation variant.
  • variants (list or tuple) – The list of available variants.
  • lift (float (default=0.0)) – The amount of uplift.
cprior.cdist.utils.check_mv_models(refclass, models)

Check models for Multivariate testing.

Parameters:
  • refclass (object) – Reference class.
  • models (dict) – Dictionary of model instances to be checked.