pythonoptimizationhyperparametersrayray-tune

When using ray tune, value defined in config returns a non-float value

I'm new to use Ray Tune. I defined my ray config as below:

ray_config = {
        "estimator/dropout_rate": tune.uniform(0.0, 0.3),
        "estimator/d_model": tune.choice([64]),
        "estimator/num_encoder_layers": tune.choice([3]),
        "estimator/context_length": tune.choice([4, 8, 12]),
        "trainer/batch_size": tune.choice([256]),
        "trainer/learning_rate": tune.uniform(0.0002, 0.01),
        "trainer/weight_decay": tune.uniform(1e-06, 1e-03)
    }

So, I tried to run my tune with this config.

tune.run(run_or_experiment=executor_run(hpo_params),
         metric="MAE",
         mode="max",
         num_samples=40, # number of trial models
         config=hpo_params,
         fail_fast=True)

But, when I see configure values in executor_run function, it has not a float or integer type.

'estimator/dropout_rate': <ray.tune.sample.Flo...d87cf7070>

How can I fix this? :(

Solution

  • You have to pass run_or_experiment=executor_run (without the brackets). If you keep the brackets, the function will be executed right away. What you really want to do is to pass a reference that Ray Tune can then execute on the remote workers.