8. AutoML (Optuna)

1. The Search Space

Hyperparameters are numbers we don't know (Learning Rate, Depth). We define a range to search.

python
def objective(trial):
    lr = trial.suggest_loguniform('lr', 1e-5, 1e-1)
    model = Train(lr)
    return model.accuracy()

2. Bayesian Optimization

Optuna doesn't guess randomly. It learns from previous trials to guess better.