Hyperparameter Tuning

Grid Search

Grid search tests all combinations of specified parameter values. Best for small parameter spaces.

-- Grid search for Random Forest
-- Parameter grid: n_trees × max_depth
SELECT neurondb.grid_search(
    'random_forest',
    'training_data',
    'features',
    'label',
    jsonb_build_object(
        'n_trees', ARRAY[10, 20, 50],
        'max_depth', ARRAY[5, 10, 15]
    )
) as best_params;

-- Tests 9 combinations (3 × 3)

Random Search

Random search samples parameter values from distributions. More efficient for large parameter spaces.

-- Random search for SVM
SELECT neurondb.random_search(
    'svm',
    'training_data',
    'features',
    'label',
    jsonb_build_object(
        'C', jsonb_build_object('type', 'uniform', 'low', 0.1, 'high', 10.0),
        'gamma', jsonb_build_object('type', 'log_uniform', 'low', 0.001, 'high', 1.0)
    ),
    10  -- Number of trials
) as best_params;

Bayesian Optimization

Uses probabilistic models to guide search toward promising parameter regions.

SELECT neurondb.bayesian_optimization(
    'random_forest',
    'training_data',
    'features',
    'label',
    jsonb_build_object(
        'n_trees', jsonb_build_object('low', 10, 'high', 100),
        'max_depth', jsonb_build_object('low', 5, 'high', 20)
    ),
    20  -- Number of iterations
) as best_params;

Next Steps

• GPU Acceleration - Speed up training
• Unified ML API - Consistent training interface
• Performance - Optimize model performance