Hyper-Parameter Optimization with SigOpt

You can use SigOpt to optimize any parameter than can be specified as a command line parameter in DeepDIVA (see Customizing Experiments).

To use SigOpt, you must specify:

  • sig-opt-token specifies the API token for your SigOpt account
  • sig-opt-runs specifies how many optimization runs to re-run the experiment on
  • sig-opt specifies the path to a text file containing a dictionary with SigOpt parameter values

For example

python template/RunMe.py --dataset-folder toy_dataset/MNIST --sig-opt-token <API_token> --sig-opt-runs 10 --sig-opt <path_to_sig_opt_text_file>

A SigOpt text file should be structured as a list of dictionaries, with a dictionary per parameter. For example, to optimize model-name and lr, the SigOpt text file should look like:

[
{"name": "model-name","type": "categorical","categorical_values": ["resnet18", "resnet152"]},
{"name": "lr", "type": "double", "bounds": { "min" : 0.001, "max": 0.01 }}
]

For more information, refer to the SigOpt Docs.