⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7fb89d4305e0>", line 36, in evaluate
File "/home/runner/work/OmniOpt/OmniOpt/.omniopt.py", line 4803, in evaluate
final_result = _run_multi_evaluation(parameters, program_string, nr_evals, trial_index, submit_time, queue_time, return_in_case_of_error)
File "<@beartype(__main__._run_multi_evaluation) at 0x7fb89d430550>", line 178, in _run_multi_evaluation
beartype.roar.BeartypeCallHintParamViolation: Function __main__._run_multi_evaluation() parameter return_in_case_of_error={'RESULT1': 99999999999999999999999999999999999999999999999999999999999, 'RESULT2': -9999999...} violates type hint dict[str, typing.Union[float, typing.Tuple[float, float]]], as dict key str 'RESULT1' value int 99999999999999999999999999999999999999999999999999999999999 not tuple or float.
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73923/73923_0_log.err
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73923/73923_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7fb89d4305e0>", line 36, in evaluate
File "/home/runner/work/OmniOpt/OmniOpt/.omniopt.py", line 4803, in evaluate
final_result = _run_multi_evaluation(parameters, program_string, nr_evals, trial_index, submit_time, queue_time, return_in_case_of_error)
File "<@beartype(__main__._run_multi_evaluation) at 0x7fb89d430550>", line 178, in _run_multi_evaluation
beartype.roar.BeartypeCallHintParamViolation: Function __main__._run_multi_evaluation() parameter return_in_case_of_error={'RESULT1': 99999999999999999999999999999999999999999999999999999999999, 'RESULT2': -9999999...} violates type hint dict[str, typing.Union[float, typing.Tuple[float, float]]], as dict key str 'RESULT1' value int 99999999999999999999999999999999999999999999999999999999999 not tuple or float.
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73935/73935_0_log.err
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73935/73935_0_log.out
Error in get_calculated_frontier: No full data points with both objectives found in /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0.
Best RESULT1 could not be determined
========================================================================
Current git-hash: 7c865634729b0a5dfd56997675d509d78a2cbb07 (last fully tested stable version 21 commits ago [1431d695e1d9911d740a43d6b86abbd58f76a3b1, 9230])
Using coverage run -p because $RUN_WITH_COVERAGE is set
[WARNING 02-19 11:08:04] ax.service.utils.with_db_settings_base: Ax currently requires a sqlalchemy version below 2.0. This will be addressed in a future release. Disabling SQL storage in Ax for now, if you would like to use SQL storage please install Ax with mysql extras via `pip install ax-platform[mysql]`.
Run-UUID: 37f85424-c402-4ba3-894b-72150ce73d1b
______ .___ ___. .__ __. __ ______ .______ .___________.___
/ __ \ | \/ | | \ | | | | / __ \ | _ \ | |__ \
| | | | | \ / | | \| | | | | | | | | |_) | `---| |----` ) |
| | | | | |\/| | | . ` | | | | | | | | ___/ | | / /
| `--' | | | | | | |\ | | | | `--' | | | | | / /_
\______/ |__| |__| |__| \__| |__| \______/ | _| |__| |____|
omniopt --live_share --send_anonymized_usage_stats --partition alpha --experiment_name=multiple_evals_per_arm --mem_gb=4 --time=1200 --worker_timeout=5 --max_eval 2 --num_parallel_jobs 1 --gpus=0 --run_program Li8udGVzdHMvb3B0aW1pemF0aW9uX2V4YW1wbGUgIC0taW50X3BhcmFtPSclKGludF9wYXJhbSknIC0tZmxvYXRfcGFyYW09JyUoZmxvYXRfcGFyYW0pJyAtLWNob2ljZV9wYXJhbT0nJShjaG9pY2VfcGFyYW0pJyAtLWludF9wYXJhbV90d289JyUoaW50X3BhcmFtX3R3byknIC0tbnJfcmVzdWx0cz0y --parameter int_param range -100 10 int --parameter float_param range -100 10 float --parameter choice_param choice 1,2,4,8,16,hallo --parameter int_param_two range -100 10 int --num_random_steps 1 --model BOTORCH_MODULAR --auto_exclude_defective_hosts --result_names RESULT1=min RESULT2=max --generate_all_jobs_at_once --revert_to_random_when_seemingly_exhausted --nr_evals_per_arm=2 --show_generate_time_table
Run-folder: /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0See https://imageseg.scads.de/omniax/share?user_id=runner&experiment_name=multiple_evals_per_arm&run_nr=1 for live-results.You have 4 CPUs available for the main process.No CUDA devices found.Generation strategy: SOBOL for 1 step and then BOTORCH_MODULAR for 1 step.
Run-Program: ./.tests/optimization_example --int_param='%(int_param)' --float_param='%(float_param)' --choice_param='%(choice_param)' --int_param_two='%(int_param_two)' --nr_results=2
Experiment parameters
┏━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃Name ┃Type ┃Lower bound┃Upper bound┃Values ┃Type ┃Log Scale?┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│int_param │range │-100 │10 ││int │No │
│float_param │range │-100 │10 ││float│No │
│choice_param │choice│││1, 2, 4, 8, 16, hallo│││
│int_param_two│range │-100 │10 ││int │No │
└───────────────┴────────┴─────────────┴─────────────┴───────────────────────┴───────┴────────────┘
Result-Names
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃Result-Name┃Min or max?┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│RESULT1 │ min│
│RESULT2 │ max│
└─────────────┴─────────────┘
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7fb89d4305e0>", line 36, in evaluate
File "/home/runner/work/OmniOpt/OmniOpt/.omniopt.py", line 4803, in evaluate
final_result = _run_multi_evaluation(parameters, program_string, nr_evals, trial_index, submit_time, queue_time, return_in_case_of_error)
File "<@beartype(__main__._run_multi_evaluation) at 0x7fb89d430550>", line 178, in _run_multi_evaluation
beartype.roar.BeartypeCallHintParamViolation: Function __main__._run_multi_evaluation() parameter return_in_case_of_error={'RESULT1': 99999999999999999999999999999999999999999999999999999999999, 'RESULT2': -9999999...} violates type hint dict[str, typing.Union[float, typing.Tuple[float, float]]], as dict key str 'RESULT1' value int 99999999999999999999999999999999999999999999999999999999999 not tuple or float.
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73923/73923_0_log.err
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73923/73923_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7fb89d4305e0>", line 36, in evaluate
File "/home/runner/work/OmniOpt/OmniOpt/.omniopt.py", line 4803, in evaluate
final_result = _run_multi_evaluation(parameters, program_string, nr_evals, trial_index, submit_time, queue_time, return_in_case_of_error)
File "<@beartype(__main__._run_multi_evaluation) at 0x7fb89d430550>", line 178, in _run_multi_evaluation
beartype.roar.BeartypeCallHintParamViolation: Function __main__._run_multi_evaluation() parameter return_in_case_of_error={'RESULT1': 99999999999999999999999999999999999999999999999999999999999, 'RESULT2': -9999999...} violates type hint dict[str, typing.Union[float, typing.Tuple[float, float]]], as dict key str 'RESULT1' value int 99999999999999999999999999999999999999999999999999999999999 not tuple or float.
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73935/73935_0_log.err
- /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73935/73935_0_log.outError in get_calculated_frontier: No full data points with both objectives found in /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0.
The data seems to be empty. Cannot plot pareto frontier.
Failed Jobs parameters
┏━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃param1┃param2┃param3┃error_description ┃
┡━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│value1│value2│value3│Some error occurred during execution (this is not a real error!).│
└────────┴────────┴────────┴───────────────────────────────────────────────────────────────────┘
Best RESULT1 could not be determinedModel generation times
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃Iteration┃Seconds┃Jobs┃Time per job┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│ 1 │ 0.029 │ 1 │ 0.029 │
│ 2 │ 0.038 │ 1 │ 0.038 │
├───────────┼─────────┼──────┼──────────────┤
│ Average │ 0.034 │ │ │
│ Median │ 0.034 │ │ │
│ Total │ 0.067 │ │ │
│ Max │ 0.038 │ │ │
│ Min │ 0.029 │ │ │
└───────────┴─────────┴──────┴──────────────┘
Job submission durations
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃ Batch┃Seconds┃Jobs┃Time per job┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│ 1 │ 0.106 │ 1 │ 0.106 │
│ 2 │ 0.048 │ 1 │ 0.048 │
├─────────┼─────────┼──────┼──────────────┤
│ Average │ 0.077 │ │ │
│ Median │ 0.077 │ │ │
│ Total │ 0.155 │ │ │
│ Max │ 0.106 │ │ │
│ Min │ 0.048 │ │ │
└─────────┴─────────┴──────┴──────────────┘
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, Started OmniOpt2 run...
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, getting new HP set (no sbatch)
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, requested 1 jobs, got 1, 0.03 s/job
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, eval #1/1 start
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, starting new job
2026-02-19 11:08:08 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, started new job
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, job_failed
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, finishing jobs, finished 1 job
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, getting new HP set (no sbatch)
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, requested 1 jobs, got 1, 0.04 s/job
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, eval #1/1 start
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, starting new job
2026-02-19 11:08:10 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, started new job
2026-02-19 11:08:12 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 1, job_failed
2026-02-19 11:08:13 (814001d5-a108-4da5-a568-bbe6e239911f): SOBOL, failed: 2, finishing jobs, finished 1 job
Job submission durations
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃ Batch ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│ 1 │ 0.106 │ 1 │ 0.106 │
│ 2 │ 0.048 │ 1 │ 0.048 │
├─────────┼─────────┼──────┼──────────────┤
│ Average │ 0.077 │ │ │
│ Median │ 0.077 │ │ │
│ Total │ 0.155 │ │ │
│ Max │ 0.106 │ │ │
│ Min │ 0.048 │ │ │
└─────────┴─────────┴──────┴──────────────┘
Model generation times
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃ Iteration ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│ 1 │ 0.029 │ 1 │ 0.029 │
│ 2 │ 0.038 │ 1 │ 0.038 │
├───────────┼─────────┼──────┼──────────────┤
│ Average │ 0.034 │ │ │
│ Median │ 0.034 │ │ │
│ Total │ 0.067 │ │ │
│ Max │ 0.038 │ │ │
│ Min │ 0.029 │ │ │
└───────────┴─────────┴──────┴──────────────┘
submitit INFO (2026-02-19 11:08:08,779) - Starting with JobEnvironment(job_id=73923, hostname=runnervmn36qa, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2026-02-19 11:08:08,780) - Loading pickle: /home/runner/work/OmniOpt/OmniOpt/runs/multiple_evals_per_arm/0/single_runs/73923/73923_submitted.pkl
Trial-Index: 0
submitit ERROR (2026-02-19 11:08:10,195) - Submitted job triggered an exception
submitit ERROR (2026-02-19 11:08:10,195) - Submitted job triggered an exception
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 69, in process_job
raise error
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/runner/.omniax_venvs/Python_3.10.12/x86_64/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7fb89d4305e0>", line 36, in evaluate
File "/home/runner/work/OmniOpt/OmniOpt/.omniopt.py", line 4803, in evaluate
final_result = _run_multi_evaluation(parameters, program_string, nr_evals, trial_index, submit_time, queue_time, return_in_case_of_error)
File "<@beartype(__main__._run_multi_evaluation) at 0x7fb89d430550>", line 178, in _run_multi_evaluation
beartype.roar.BeartypeCallHintParamViolation: Function __main__._run_multi_evaluation() parameter return_in_case_of_error={'RESULT1': 99999999999999999999999999999999999999999999999999999999999, 'RESULT2': -9999999...} violates type hint dict[str, typing.Union[float, typing.Tuple[float, float]]], as dict key str 'RESULT1' value int 99999999999999999999999999999999999999999999999999999999999 not tuple or float.