ERROR while trying to mark job as failure: Can only mark failed a trial that is currently running.
โ Exception in parallel job handling: Can only mark failed a trial that is currently running.
It seems like the search space was exhausted. You were able to get 8% of the jobs you requested (got: 2, submitted: 3, failed: 0, requested: 23) after main ran
========================================================================
Current git-hash: c120532e609120a644cc73615366e807cf34d2bd (last fully tested stable version 3 commits ago [9043d91dbe370183252bdf120ecb2c226402839d, 8689])
Using coverage run -p because $RUN_WITH_COVERAGE is set
[WARNING 10-21 07:23:38] ax.service.utils.with_db_settings_base: Ax currently requires a sqlalchemy version below 2.0. This will be addressed in a future release. Disabling SQL storage in Ax for now, if you would like to use SQL storage please install Ax with mysql extras via `pip install ax-platform[mysql]`.
Run-UUID: 9f76b6b6-e74e-43cb-9ead-75a89e3e3510
____ _ ____ _ ___
/ __ \ (_)/ __ \ | | |__ \
| | | |_ __ ___ _ __ _| | | |_ __ | |_ ) |
| | | | '_ ` _ \| '_ \| | | | | '_ \| __| / /
| |__| | | | | | | | | | | |__| | |_) | |_ / /_
\____/|_| |_| |_|_| |_|_|\____/| .__/ \__|____|
| |
|_|
omniopt --max_nr_of_zero_results 3 --live_share --send_anonymized_usage_stats --partition=alpha --gpus=0 --experiment_name=test_wronggoing_stuff --mem_gb=1 --time=120 --worker_timeout=1 --max_eval=23 --num_parallel_jobs=10 --run_program=Li8udGVzdHMvdGVzdF93cm9uZ2dvaW5nX3N0dWZmLmJpbi9iaW4vJShwcm9ncmFtKQ== --parameter program choice divide_by_0,simple_ok,perl,exit_code_137,result_but_exit_code_stdout_stderr,exit_code_stdout,signal_but_has_output,exit_code_no_output,exit_code_stdout_stderr,module_not_found,signal,empty,file_does_not_exist,sleep_10_and_result_exit_252,no_chmod_x,no_shebang,perl_module_fail,segfault,wrong_arch,python_name_error,syntax_error,tensorflow_syntax_error,force_keyboard_interrupt --num_random_steps=1 --follow
Run-folder: /home/runner/work/OmniOpt/OmniOpt/runs/test_wronggoing_stuff/0See https://imageseg.scads.de/omniax/share?user_id=runner&experiment_name=test_wronggoing_stuff&run_nr=4 for live-results.You have 4 CPUs available for the main process.No CUDA devices found.Generation strategy: SOBOL for 1 step and then BOTORCH_MODULAR for 22 steps.
Run-Program: ./.tests/test_wronggoing_stuff.bin/bin/%(program)
Experiment parameters
โโโโโโโโโโโณโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โName โType โValues โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โprogramโchoiceโdivide_by_0, simple_ok, perl, exit_code_137, result_but_exit_code_stdout_stderr, exit_code_stdout, signal_but_has_output, exit_code_no_output, exit_code_stdout_stderr, โ
โโโmodule_not_found, signal, empty, file_does_not_exist, sleep_10_and_result_exit_252, no_chmod_x, no_shebang, perl_module_fail, segfault, wrong_arch, python_name_error, โ
โโโsyntax_error, tensorflow_syntax_error, force_keyboard_interrupt โ
โโโโโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Result-Names
โโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โResult-NameโMin or max?โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โRESULT โ minโ
โโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
ERROR while trying to mark job as failure: Can only mark failed a trial that is currently running.โ Exception in parallel job handling: Can only mark failed a trial that is currently running.
It seems like the search space was exhausted. You were able to get 8% of the jobs you requested (got: 2, submitted: 3, failed: 0, requested: 23) after main ran Failed Jobs parameters
โโโโโโโโโโณโโโโโโโโโณโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โparam1โparam2โparam3โerror_description โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โvalue1โvalue2โvalue3โSome error occurred during execution (this is not a real error!).โ
โโโโโโโโโโดโโโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Best RESULT, min (total: 2)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโ
โprogram โRESULTโ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โresult_but_exit_code_stdout_stderrโ7564.0โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโ
Not printing sixel graphics in CI
Runtime Infos
โโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโณโโโโโโโโโโโโณโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โNumber of evaluationsโMin time โMax time โAverage timeโMedian timeโ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ 3 โ 6.00 sec โ 6.00 sec โ 6.00 sec โ 6.00 sec โ
โโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, Started OmniOpt2 run...
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, getting new HP set (no sbatch)
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, requested 1 jobs, got 1, 0.04 s/job
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, eval #1/1 start
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, starting new job
2025-10-21 07:23:40 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, started new job
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, job_failed
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, getting new HP set (no sbatch)
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, requested 1 jobs, got 1, 0.03 s/job
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, eval #1/1 start
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, starting new job
2025-10-21 07:23:49 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, started new job
2025-10-21 07:23:58 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, new result: 7564.0
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, finishing jobs, finished 1 job
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, requested 1 jobs, got 1, 0.31 s/job
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, eval #1/1 start
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, starting new job
2025-10-21 07:23:59 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, started new job
2025-10-21 07:24:08 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, new result: 25731.0
2025-10-21 07:24:09 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, finishing jobs, finished 1 job
2025-10-21 07:24:10 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:10 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:11 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:11 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:12 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:12 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:12 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:13 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:13 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:14 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:14 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:15 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:15 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:15 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:16 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:16 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:17 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:17 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:18 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:18 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:18 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, requested 1 jobs, got 0, n/a
2025-10-21 07:24:18 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, found 1 zero-jobs (max: 3)
2025-10-21 07:24:18 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:19 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:19 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:20 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:20 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:20 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:21 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:21 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:22 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:22 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:23 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:23 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:23 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:24 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:24 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:25 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:25 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:25 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:26 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:26 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:26 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, requested 1 jobs, got 0, n/a
2025-10-21 07:24:26 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, found 2 zero-jobs (max: 3)
2025-10-21 07:24:27 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:27 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:28 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:28 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:28 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:29 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:29 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:30 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:30 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:31 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:31 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:31 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:32 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:32 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:33 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:33 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:34 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:34 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:34 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:35 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, getting new HP set (no sbatch)
2025-10-21 07:24:35 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, requested 1 jobs, got 0, n/a
2025-10-21 07:24:35 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, found 3 zero-jobs (max: 3)
2025-10-21 07:24:35 (99b4034f-9556-4d75-af88-170bbef11641): SOBOL, best RESULT: 7564, 3 empty jobs (>= 3)
Job submission durations
โโโโโโโโโโโณโโโโโโโโโโณโโโโโโโณโโโโโโโโโโโโโโโ
โ Batch โ Seconds โ Jobs โ Time per job โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ 1 โ 0.096 โ 1 โ 0.096 โ
โ 2 โ 0.035 โ 1 โ 0.035 โ
โ 3 โ 0.064 โ 1 โ 0.064 โ
โโโโโโโโโโโผโโโโโโโโโโผโโโโโโโผโโโโโโโโโโโโโโโค
โ Average โ 0.065 โ โ โ
โ Median โ 0.064 โ โ โ
โ Total โ 0.194 โ โ โ
โ Max โ 0.096 โ โ โ
โ Min โ 0.035 โ โ โ
โโโโโโโโโโโดโโโโโโโโโโดโโโโโโโดโโโโโโโโโโโโโโโ