OmniOpt2-Logo ScaDS.ai-Logo

🌳 Tree-structured Parzen Estimator (TPE)

What is TPE?

The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization algorithm commonly used for hyperparameter tuning. Instead of modeling the objective function directly, it models the probability densities of good and bad hyperparameter configurations separately, allowing efficient sampling of promising configurations.

How does it operate?

This approach enables efficient exploration of complex, high-dimensional, and conditional search spaces often encountered in machine learning hyperparameter tuning.

How does TPE work in practice?

TPE works by modeling the likelihood of hyperparameters leading to good versus bad outcomes, rather than directly modeling the objective function. At each step, it selects new hyperparameter candidates by maximizing the expected improvement, guided by its density models. This results in a focused and efficient search process that adapts as more data becomes available.

When to use TPE

When not to use TPE