To cancel, press CTRLc, then run 'scancel 614219'
โ Importing logging...โ Importing warnings...โ Importing argparse...โ Importing datetime...โ Importing dataclass...โ Importing hashlib...โ Importing socket...โ Importing stat...โ Importing pwd...โ Importing signal...โ Importing base64...โ Importing json...โ Importing yaml...โ Importing toml...โ Importing csv...โ Importing ast...โ Importing rich.table...โ Importing rich print...โ Importing rich.pretty...โ Importing rich.prompt...โ Importing types.FunctionType...โ Importing typing...โ Importing ThreadPoolExecutor...โ Importing submitit.LocalExecutor...โ Importing submitit.Job...โ Importing importlib.util...โ Importing inspect...โ Importing platform...โ Importing inspect frame info...โ Importing pathlib.Path...โ Importing uuid...โ Importing traceback...โ Importing cowsay...โ Importing psutil...โ Importing shutil...โ Importing itertools.combinations...โ Importing os.listdir...โ Importing os.path...โ Importing PIL.Image...โ Importing sixel...โ Importing subprocess...โ Importing tqdm...โ ฆImporting beartype...โ Importing statistics...โ Trying to import pyfiglet...โ งImporting helpers...โ Parsing arguments...โ Importing torch...โ Importing numpy...โ Importing collections...โ ฆImporting ax...โ Importing ax.core.generator_run...โ Importing Cont_X_trans and Y_trans from ax.modelbridge.registry...โ Importing ax.core.arm...โ Importing ax.core.objective...โ Importing ax.core.Metric...โ Importing ax.exceptions.core...โ Importing ax.exceptions.generation_strategy...โ Importing CORE_DECODER_REGISTRY...โ Trying ax.generation_strategy.generation_node...โ Importing GenerationStep, GenerationStrategy from generation_strategy...โ Importing GenerationNode from generation_node...โ Importing ExternalGenerationNode...โ Importing MaxTrials...โ Importing GeneratorSpec...โ Importing Models from ax.modelbridge.registry...โ Importing get_pending_observation_features...โ Importing load_experiment...โ Importing save_experiment...โ Importing save_experiment_to_db...โ Importing TrialStatus...โ Importing Data...โ Importing Experiment...โ Importing parameter types...โ Importing TParameterization...โ Importing pandas...โ Importing AxClient and ObjectiveProperties...โ Importing RandomForestRegressor...โ Importing botorch...โ Importing submitit...โ Importing ax logger...โ Importing SQL-Storage-Stuff...
Run-UUID: f0934adb-9ed7-4007-a874-dd82b5ccd578
.oOOOo. .oOOOo.
.O o. o .O o. .oOOo.
O o O o O O
o O o O oOo o
O o `oOOoOO. 'OoOo. O O o .oOo. o O'
o O O o o o O o o O O o O O
`o O' o O O O o O `o O' o O o .O
`OoooO' O o o o O o' `OoooO' oOoO' `oO oOoOoO
O
o'
โ Writing worker creation log...
omniopt --partition is_ignored_here --experiment_name=slurm_in_docker_test --mem_gb=1 --time=60 --worker_timeout=1 --max_eval=100000 --num_parallel_jobs=1 --run_program ZWNobyAiUkVTVUxUOiAlKGludF9wYXJhbSklKGludF9wYXJhbV90d28pJShmbG9hdF9wYXJhbSki --parameter int_param range -100 10 int --parameter float_param range -100 10 float --parameter int_param_two range -100 10 int --follow --num_random_steps 2000 --model BOTORCH_MODULAR --auto_exclude_defective_hosts --max_nr_of_zero 3 --show_generate_time_table --show_generation_and_submission_sixel --no_sleep --number_of_generators 5 --live_share --gpus=1
โ Disabling logging...โ Setting run folder...โ Creating folder /home/h8/s3811141/test/randomtest_93382/OmniOpt/runs/slurm_in_docker_test/3...โ Writing revert_to_random_when_seemingly_exhausted file ...โ Writing username state file...โ Writing result names file...โ Writing result min/max file...โ Saving state files...Run-folder: /home/h8/s3811141/test/randomtest_93382/OmniOpt/runs/slurm_in_docker_test/3โ Printing run info...โ Initializing NVIDIA-Logs...โ Writing ui_url file if it is present...โ Writing live_share file if it is present...โ Writing job_start_time file...โ Writing git info file...โ Checking max_eval...โ Calculating number of steps...โ Adding excluded nodes...โ Handling random steps...โ Initializing ax_client...
[WARNING 08-12 16:06:54] ax.service.ax_client: Selecting a GenerationStrategy when using BatchTrials is in beta. Double check the recommended strategy matches your expectations.
โ Setting orchestrator...You have 1 CPUs available for the main process.Using CUDA device NVIDIA H100.Generation strategy: SOBOL for 2000 steps and then BOTORCH_MODULAR for 98000 steps.
Run-Program: echo "RESULT: %(int_param)%(int_param_two)%(float_param)"
Experiment parameters
โโโโโโโโโโโโโโโโโณโโโโโโโโณโโโโโโโโโโโโโโณโโโโโโโโโโโโโโณโโโโโโโโณโโโโโโโโโโโโโ
โName โType โLower boundโUpper boundโType โLog Scale?โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โint_param โrangeโ-100 โ10 โint โNo โ
โfloat_param โrangeโ-100 โ10 โfloatโNo โ
โint_param_twoโrangeโ-100 โ10 โint โNo โ
โโโโโโโโโโโโโโโโโดโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโดโโโโโโโโโโโโโ
Result-Names
โโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โResult-NameโMin or max?โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โRESULT โ minโ
โโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
Failed to start worker 1: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with errorFailed to start worker 2: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with errorFailed to start worker 3: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with errorFailed to start worker 4: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with error