To cancel, press CTRL c, then run 'scancel 1782551'
โ Importing logging...
โ Importing warnings...
โ Importing argparse...
โ Importing datetime...
โ Importing dataclass...
โ Importing hashlib...
โ Importing socket...
โ Importing stat...
โ Importing pwd...
โ Importing signal...
โ Importing base64...
โ Importing json...
โ ด Importing yaml...
โ Importing toml...
โ Importing csv...
โ Importing ast...
โ Importing rich.table...
โ Importing rich print...
โ Importing rich.pretty...
โ Importing rich.prompt...
โ Importing types.FunctionType...
โ Importing typing...
โ Importing ThreadPoolExecutor...
โ ผ Importing submitit.LocalExecutor...
โ Importing submitit.Job...
โ Importing importlib.util...
โ Importing inspect...
โ Importing platform...
โ Importing inspect frame info...
โ Importing pathlib.Path...
โ Importing uuid...
โ Importing traceback...
โ Importing cowsay...
โ Importing psutil...
โ Importing shutil...
โ Importing itertools.combinations...
โ Importing os.listdir...
โ Importing os.path...
โ ผ Importing PIL.Image...
โ ธ Importing sixel...
โ Importing subprocess...
โ Importing tqdm...
โ Importing beartype...
โ Importing statistics...
โ Trying to import pyfiglet...
โ ธ Importing helpers...
โ Parsing arguments...
โ Importing torch...
โ Importing numpy...
โ Importing collections...
โ Importing ax...
โ Importing ax.core.generator_run...
โ Importing Cont_X_trans and Y_trans from ax.modelbridge.registry...
โ Importing ax.core.arm...
โ Importing ax.core.objective...
โ Importing ax.core.Metric...
โ Importing ax.exceptions.core...
โ Importing ax.exceptions.generation_strategy...
โ Importing CORE_DECODER_REGISTRY...
โ Trying ax.generation_strategy.generation_node...
โ Importing GenerationStep, GenerationStrategy from generation_strategy...
โ Importing GenerationNode from generation_node...
โ Importing ExternalGenerationNode...
โ Importing MaxTrials...
โ Importing GeneratorSpec...
โ Importing Models from ax.modelbridge.registry...
โ Importing get_pending_observation_features...
โ Importing load_experiment...
โ Importing save_experiment...
โ Importing save_experiment_to_db...
โ Importing TrialStatus...
โ Importing Data...
โ Importing Experiment...
โ Importing parameter types...
โ Importing TParameterization...
โ Importing pandas...
โ Importing AxClient and ObjectiveProperties...
โ Importing RandomForestRegressor...
โ Importing botorch...
โ Importing submitit...
โ Importing ax logger...
โ Importing SQL-Storage-Stuff...
Run-UUID: 0d9e1a98-51fa-4312-80ff-e1385ab32279
____ _ ____ _ ___
/ __ \ (_)/ __ \ | | |__ \
| | | |_ __ ___ _ __ _| | | |_ __ | |_ ) |
| | | | '_ ` _ \| '_ \| | | | | '_ \| __| / /
| |__| | | | | | | | | | | |__| | |_) | |_ / /_
\____/|_| |_| |_|_| |_|_|\____/| .__/ \__|____|
| |
|_|
โ Writing worker creation log...
omniopt --partition is_ignored_here --experiment_name=slurm_in_docker_test --mem_gb=1 --time=60 --worker_timeout=1 --max_eval=100000 --num_parallel_jobs=1 --run_program ZWNobyAiUkVTVUxUOiAlKGludF9wYXJhbSklKGludF9wYXJhbV90d28pJShmbG9hdF9wYXJhbSki --parameter int_param range -100 10 int --parameter float_param range -100 10 float --parameter int_param_two range -100 10 int --follow --num_random_steps 2000 --model BOTORCH_MODULAR --auto_exclude_defective_hosts --max_nr_of_zero 3 --show_generate_time_table --show_generation_and_submission_sixel --no_sleep --number_of_generators 5 --live_share --gpus=1
โ Disabling logging...
โ Setting run folder...
โ Creating folder /home/h8/s3811141/test/randomtest_93382/OmniOpt/runs/slurm_in_docker_test/2...
โ Writing revert_to_random_when_seemingly_exhausted file ...
โ Writing username state file...
โ Writing result names file...
โ Writing result min/max file...
โ Saving state files...
Run-folder: /home/h8/s3811141/test/randomtest_93382/OmniOpt/runs/slurm_in_docker_test/2
โ Printing run info...
โ Initializing NVIDIA-Logs...
โ Writing ui_url file if it is present...
โ Writing live_share file if it is present...
โ Writing job_start_time file...
โ ด Writing git info file...
โ Checking max_eval...
โ Calculating number of steps...
โ Adding excluded nodes...
โ Handling random steps...
โ Initializing ax_client...
[WARNING 08-12 16:05:39] ax.service.ax_client: Selecting a GenerationStrategy when using BatchTrials is in beta. Double check the recommended strategy matches your expectations.
โ Setting orchestrator...
You have 2 CPUs available for the main process. Using CUDA device NVIDIA A100-SXM4-40GB. Generation strategy: SOBOL for 2000 steps and then BOTORCH_MODULAR for 98000 steps.
Run-Program: echo "RESULT: %(int_param)%(int_param_two)%(float_param)"
Experiment parameters
โโโโโโโโโโโโโโโโโณโโโโโโโโณโโโโโโโโโโโโโโณโโโโโโโโโโโโโโณโโโโโโโโณโโโโโโโโโโโโโ
โ Name โ Type โ Lower bound โ Upper bound โ Type โ Log Scale? โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ int_param โ range โ -100 โ 10 โ int โ No โ
โ float_param โ range โ -100 โ 10 โ float โ No โ
โ int_param_two โ range โ -100 โ 10 โ int โ No โ
โโโโโโโโโโโโโโโโโดโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโดโโโโโโโโโโโโโ
Result-Names
โโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Result-Name โ Min or max? โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ RESULT โ min โ
โโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
Failed to start worker 1: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with error
Failed to start worker 2: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with error
Failed to start worker 3: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with error
Failed to start worker 4: sbatch: lua: Please always specify the node count using either the parameter -N <#nodes> or --nodes=<#nodes>.
sbatch: error: cli_filter plugin terminated with error
timestamp,ram_usage_mb,cpu_usage_percent
1755007546,678.91015625,8.9