bayex.acq module

bayex.acq module#

This module contains the implementation of various acquisition functions used in Bayesian optimization. These functions help in selecting the next point to sample based on the current model of the objective function.

expected_improvement

Expected Improvement (EI) acquisition function.

probability_improvement

Probability of Improvement (PI) acquisition function.

upper_confidence_bounds

Upper Confidence Bound (UCB) acquisition function.

lower_confidence_bounds

Lower Confidence Bound (LCB) acquisition function.
bayex.acq.expected_improvement(x_pred, xs, ys, mask, gp_params, xi=0.01)

Expected Improvement (EI) acquisition function.

Favors points with high improvement over the current best observed value, balancing exploitation and exploration.

The formula is:

\[EI(x) = (\mu(x) - y^* - \xi) \Phi(z) + \sigma(x) \phi(z)\]

where:

\[z = \frac{\mu(x) - y^* - \xi}{\sigma(x)}\]
Parameters:

  • x_pred (Array) – Candidate input locations to evaluate.

  • xs (Array) – Observed inputs.

  • ys (Array) – Observed function values.

  • mask (Array) – Boolean mask indicating valid entries in ys.

  • gp_params (GPParams) – Gaussian Process hyperparameters.

  • xi (float) – Exploration-exploitation tradeoff parameter.

Returns:

EI scores at x_pred.

bayex.acq.lower_confidence_bounds(x_pred, xs, ys, mask, gp_params, kappa=2.576)

Lower Confidence Bound (LCB) acquisition function.

Useful for minimization tasks. Encourages sampling in uncertain regions with low predicted values.

The formula is:

\[LCB(x) = \mu(x) - \kappa \cdot \sigma(x)\]
Parameters:

  • x_pred (Array) – Candidate input locations to evaluate.

  • xs (Array) – Observed inputs.

  • ys (Array) – Observed function values.

  • mask (Array) – Boolean mask indicating valid entries in ys.

  • gp_params (GPParams) – Gaussian Process hyperparameters.

  • kappa (float) – Weighting factor for uncertainty.

Returns:

LCB scores at x_pred.

bayex.acq.probability_improvement(x_pred, xs, ys, mask, gp_params, xi=0.01)

Probability of Improvement (PI) acquisition function.

Estimates the probability that a candidate point will improve over the current best observed value.

The formula is:

\[PI(x) = \Phi\left(\frac{\mu(x) - y^* - \xi}{\sigma(x)}\right)\]
Parameters:

  • x_pred (Array) – Candidate input locations to evaluate.

  • xs (Array) – Observed inputs.

  • ys (Array) – Observed function values.

  • mask (Array) – Boolean mask indicating valid entries in ys.

  • gp_params (GPParams) – Gaussian Process hyperparameters.

  • xi (float) – Improvement margin for sensitivity.

Returns:

PI scores at x_pred.

bayex.acq.upper_confidence_bounds(x_pred, xs, ys, mask, gp_params, kappa=0.01)

Upper Confidence Bound (UCB) acquisition function.

Promotes exploration by favoring points with high predictive uncertainty.

The formula is:

\[UCB(x) = \mu(x) + \kappa \cdot \sigma(x)\]
Parameters:

  • x_pred (Array) – Candidate input locations to evaluate.

  • xs (Array) – Observed inputs.

  • ys (Array) – Observed function values.

  • mask (Array) – Boolean mask indicating valid entries in ys.

  • gp_params (GPParams) – Gaussian Process hyperparameters.

  • kappa (float) – Weighting factor for uncertainty.

Returns:

UCB scores at x_pred.