Pre-training

sindae.algorithms.pretrain

Supervised pre-training of the MLP on the smoother arrays before the main training loop. This gives both solvers a sensible starting point and is applicable to the simultaneous and decomposition approaches alike.

• PretrainConfig — the few SGD hyperparameters (epochs, batch size, regularization).

• pretrain_mlp — fits the network to the (nn_input, nn_output) pairs in an InstanceData, applying input/output normalization internally.

Usage¶

from sindae import extract_instance_data
from sindae.algorithms.smoother import solve_smoother
from sindae.algorithms.pretrain import PretrainConfig, pretrain_mlp

# 1. Solve the smoother to get smooth (x, z) trajectories ...
smoother_m    = solve_smoother(problem, mlp, smooth_coef=1.0)
smoother_data = extract_instance_data(problem, smoother_m)

# 2. ... then pre-train the network on those pairs.
mlp = pretrain_mlp(
    mlp, smoother_data,
    PretrainConfig(epochs=200, batch_size=32, reg_coef=1e-3),
)

The returned mlp is ready to pass to solve_simultaneous or train_decomp.

API reference¶

PretrainConfig¶

class PretrainConfig(epochs: int = 200, batch_size: int = 32, reg_coef: float = 0.1)

Hyperparameters for supervised MLP pretraining on smoother arrays.

epochs=0 runs no pretraining passes (the MLP is returned unchanged), which is how HybridDAE users opt out of the pretraining stage.

Fields

• epochs (int, default 200)

• batch_size (int, default 32)

• reg_coef (float, default 0.1)

pretrain_mlp¶

pretrain_mlp(mlp: SimpleMLP, data: InstanceData, cfg: PretrainConfig) -> SimpleMLP

Pretrain the MLP on (nn_input, nn_output) pairs from data using SGD.

Normalisation is applied internally using statistics from data (data.input_mean/std, data.output_mean/std).

Parameters

• mlp (SimpleMLP)

• data (InstanceData)

• cfg (PretrainConfig)

Returns

• mlp (SimpleMLP (updated weights))