⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.out
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.err not found
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.out
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.err not found
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.out
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.err not found
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.out
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.err not found
To cancel, press CTRLc, then run 'scancel 17316646'
⠋Importing logging...⠋Importing warnings...⠋Importing argparse...⠋Importing datetime...⠋Importing dataclass...⠋Importing hashlib...⠋Importing socket...⠋Importing stat...⠋Importing pwd...⠋Importing signal...⠋Importing base64...⠋Importing json...⠋Importing yaml...⠋Importing toml...⠋Importing csv...⠋Importing ast...⠋Importing rich.progress...⠋Importing rich.table...⠋Importing rich print...⠋Importing rich.pretty...⠋Importing fcntl...⠋Importing types.FunctionType...⠋Importing typing...⠋Importing ThreadPoolExecutor...⠸Importing submitit.LocalExecutor...⠋Importing submitit.Job...⠋Importing importlib.util...⠋Importing inspect...⠋Importing platform...⠋Importing inspect frame info...⠋Importing pathlib.Path...⠋Importing uuid...⠋Importing traceback...⠋Importing cowsay...⠋Importing psutil...⠋Importing shutil...⠋Importing itertools.combinations...⠇Importing pandas...⠋Importing os.listdir...⠋Importing os.path...⠋Importing PIL.Image...⠋Importing sixel...⠋Importing subprocess...⠋Importing tqdm...⠴Importing beartype...⠋Importing statistics...⠋Trying to import pyfiglet...⠋Importing rich_argparse...⠴Importing helpers...⠋Parsing arguments...⠴Importing torch...⠋Importing numpy...⠋Importing collections...⠼Importing ax...⠋Importing ax.core.generator_run...⠋Importing ax.core.arm...⠋Importing ax.core.objective...⠋Importing ax.core.Metric...⠋Importing ax.exceptions.core...⠋Importing ax.exceptions.generation_strategy...⠋Importing CORE_DECODER_REGISTRY...⠋Trying ax.generation_strategy.generation_node...⠋Importing GenerationStep, GenerationStrategy from generation_strategy...⠋Importing GenerationNode from generation_node...⠋Importing ExternalGenerationNode...⠋Importing MaxTrials...⠋Importing GeneratorSpec...⠋Importing Models from ax.modelbridge.registry...⠋Importing get_pending_observation_features...⠋Importing load_experiment...⠋Importing save_experiment...⠋Importing TrialStatus...⠋Importing Data...⠋Importing Experiment...⠋Importing parameter types...⠋Importing TParameterization...⠋Importing AxClient and ObjectiveProperties...⠋Importing RandomForestRegressor...⠋Importing botorch...⠋Importing submitit...⠋Importing ax logger...
Run-UUID: 78ad29e9-aa92-4322-be3e-0c929e1ee632
_________________________________________
| OmniOpt2 - Animals are friends, not food! |
=========================================
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||
⠋Checking if has random steps...⠋Writing worker creation log...
omniopt --partition=barnard --experiment_name=vergl_botorch_1GB_20w --mem_gb=1 --time=600 --worker_timeout=30 --max_eval=2000 --num_parallel_jobs=20 --gpus=0 --num_random_steps=20 --follow --live_share --send_anonymized_usage_stats --result_names RESULT=min TIME=min --run_program=YmFzaCAvZGF0YS9ob3JzZS93cy9wd2lua2xlci1vb3B0L3J1bi5zaCAlKGVwb2NocykgJShscikgJShic3opICUoZGEpICUoZGIp --cpus_per_task=1 --nodes_per_job=1 --generate_all_jobs_at_once --revert_to_random_when_seemingly_exhausted --model=BOTORCH_MODULAR --n_estimators_randomforest=100 --run_mode=local --decimalrounding=4 --occ_type=euclid --main_process_gb=8 --max_nr_of_zero_results=50 --slurm_signal_delay_s=0 --max_failed_jobs=0 --max_attempts_for_generation=20 --num_restarts=20 --raw_samples=1024 --batch_limit=32 --parameter epochs range 10 350 int false --parameter lr range 0.001 0.05 float false --parameter bsz range 4 32 int false --parameter da range 10 100 int false --parameter db range 10 150 int false --ui_url aHR0cHM6Ly9pbWFnZXNlZy5zY2Fkcy5kZS9vbW5pYXgvZ3VpP3BhcnRpdGlvbj1iYXJuYXJkJmV4cGVyaW1lbnRfbmFtZT12ZXJnbF9ib3RvcmNoXzFHQl8yMHcmcmVzZXJ2YXRpb249JmFjY291bnQ9Jm1lbV9nYj0xJnRpbWU9NjAwJndvcmtlcl90aW1lb3V0PTMwJm1heF9ldmFsPTIwMDAmbnVtX3BhcmFsbGVsX2pvYnM9MjAmZ3B1cz0wJm51bV9yYW5kb21fc3RlcHM9MjAmZm9sbG93PTEmbGl2ZV9zaGFyZT0xJnNlbmRfYW5vbnltaXplZF91c2FnZV9zdGF0cz0xJmNoZWNrb3V0X3RvX2xhdGVzdF90ZXN0ZWRfdmVyc2lvbj0wJmNvbnN0cmFpbnRzPSZyZXN1bHRfbmFtZXM9UkVTVUxUJTNEbWluJTIwVElNRSUzRG1pbiZydW5fcHJvZ3JhbT1iYXNoJTIwJTJGZGF0YSUyRmhvcnNlJTJGd3MlMkZwd2lua2xlci1vb3B0JTJGcnVuLnNoJTIwJTI1KGVwb2NocyklMjAlMjUobHIpJTIwJTI1KGJzeiklMjAlMjUoZGEpJTIwJTI1KGRiKSZjcHVzX3Blcl90YXNrPTEmbm9kZXNfcGVyX2pvYj0xJnNlZWQ9JnZlcmJvc2U9MCZnZW5lcmF0ZV9hbGxfam9ic19hdF9vbmNlPTEmZmxhbWVfZ3JhcGg9MCZkZWJ1Zz0wJnJldmVydF90b19yYW5kb21fd2hlbl9zZWVtaW5nbHlfZXhoYXVzdGVkPTEmZ3JpZHNlYXJjaD0wJm1vZGVsPUJPVE9SQ0hfTU9EVUxBUiZleHRlcm5hbF9nZW5lcmF0b3I9Jm5fZXN0aW1hdG9yc19yYW5kb21mb3Jlc3Q9MTAwJmluc3RhbGxhdGlvbl9tZXRob2Q9Y2xvbmUmcnVuX21vZGU9bG9jYWwmZGVjaW1hbHJvdW5kaW5nPTQmZGlzYWJsZV90cWRtPTAmdmVyYm9zZV90cWRtPTAmZm9yY2VfbG9jYWxfZXhlY3V0aW9uPTAmYXV0b19leGNsdWRlX2RlZmVjdGl2ZV9ob3N0cz0wJnNob3dfc2l4ZWxfZ2VuZXJhbD0wJnNob3dfc2l4ZWxfdHJpYWxfaW5kZXhfcmVzdWx0PTAmc2hvd19zaXhlbF9zY2F0dGVyPTAmc2hvd193b3JrZXJfcGVyY2VudGFnZV90YWJsZV9hdF9lbmQ9MCZvY2M9MCZvY2NfdHlwZT1ldWNsaWQmbm9fc2xlZXA9MCZzbHVybV91c2Vfc3J1bj0wJnZlcmJvc2VfYnJlYWtfcnVuX3NlYXJjaF90YWJsZT0wJmFiYnJldmlhdGVfam9iX25hbWVzPTAmbWFpbl9wcm9jZXNzX2diPTgmbWF4X25yX29mX3plcm9fcmVzdWx0cz01MCZzbHVybV9zaWduYWxfZGVsYXlfcz0wJm1heF9mYWlsZWRfam9icz0wJmV4Y2x1ZGU9JnVzZXJuYW1lPSZnZW5lcmF0aW9uX3N0cmF0ZWd5PSZyb290X3ZlbnZfZGlyPSZ3b3JrZGlyPSZkb250X2ppdF9jb21waWxlPTAmZml0X291dF9vZl9kZXNpZ249MCZyZWZpdF9vbl9jdj0wJnNob3dfZ2VuZXJhdGVfdGltZV90YWJsZT0wJmRvbnRfd2FybV9zdGFydF9yZWZpdHRpbmc9MCZtYXhfYXR0ZW1wdHNfZm9yX2dlbmVyYXRpb249MjAmbnVtX3Jlc3RhcnRzPTIwJnJhd19zYW1wbGVzPTEwMjQmZm9yY2VfY2hvaWNlX2Zvcl9yYW5nZXM9MCZub190cmFuc2Zvcm1faW5wdXRzPTAmZml0X2FiYW5kb25lZD0wJm5vX25vcm1hbGl6ZV95PTAmYmF0Y2hfbGltaXQ9MzImcGFyYW1ldGVyXzBfbmFtZT1lcG9jaHMmcGFyYW1ldGVyXzBfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfMF9taW49MTAmcGFyYW1ldGVyXzBfbWF4PTM1MCZwYXJhbWV0ZXJfMF9udW1iZXJfdHlwZT1pbnQmcGFyYW1ldGVyXzBfbG9nX3NjYWxlPWZhbHNlJnBhcmFtZXRlcl8xX25hbWU9bHImcGFyYW1ldGVyXzFfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfMV9taW49MC4wMDEmcGFyYW1ldGVyXzFfbWF4PTAuMDUmcGFyYW1ldGVyXzFfbnVtYmVyX3R5cGU9ZmxvYXQmcGFyYW1ldGVyXzFfbG9nX3NjYWxlPWZhbHNlJnBhcmFtZXRlcl8yX25hbWU9YnN6JnBhcmFtZXRlcl8yX3R5cGU9cmFuZ2UmcGFyYW1ldGVyXzJfbWluPTQmcGFyYW1ldGVyXzJfbWF4PTMyJnBhcmFtZXRlcl8yX251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfMl9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzNfbmFtZT1kYSZwYXJhbWV0ZXJfM190eXBlPXJhbmdlJnBhcmFtZXRlcl8zX21pbj0xMCZwYXJhbWV0ZXJfM19tYXg9MTAwJnBhcmFtZXRlcl8zX251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfM19sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzRfbmFtZT1kYiZwYXJhbWV0ZXJfNF90eXBlPXJhbmdlJnBhcmFtZXRlcl80X21pbj0xMCZwYXJhbWV0ZXJfNF9tYXg9MTUwJnBhcmFtZXRlcl80X251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfNF9sb2dfc2NhbGU9ZmFsc2UmcGFydGl0aW9uPWJhcm5hcmQmbnVtX3BhcmFtZXRlcnM9NQ==
⠋Disabling logging...⠋Saving state files...
Run-folder: runs/vergl_botorch_1GB_20w/0
⠋Parsing parameters...⠋Initializing ax_client...mended strategy matches your expectations.
⠋Getting experiment parameters...⠋Setting orchestrator...You have 2 CPUs available for the main process.No CUDA devices found.Generation strategy: SOBOL for 20 steps and then BOTORCH_MODULAR for 1980 steps.
Run-Program: bash /data/horse/ws/pwinkler-oopt/run.sh %(epochs) %(lr) %(bsz) %(da) %(db)
Experiment parameters
┏━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃Name ┃Type ┃Lower bound┃Upper bound┃Type ┃Log Scale?┃
┡━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│epochs│range│10 │350 │int │No │
│lr │range│0.001 │0.05 │float│No │
│bsz │range│4 │32 │int │No │
│da │range│10 │100 │int │No │
│db │range│10 │150 │int │No │
└────────┴───────┴─────────────┴─────────────┴───────┴────────────┘
Result-Names
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃Result-Name┃Min or max?┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│RESULT │ min│
│TIME │ min│
└─────────────┴─────────────┘
See https://imageseg.scads.de/omniax/share?user_id=pwinkler&experiment_name=vergl_botorch_1GB_20w&run_nr=0 for live-results.
█▀▀▀▀▀█ ▄ █ ▀ ▀▄▄ █▀▀ ▄▀ █▄▄ █▀▀▀▀▀█
█ ███ █ ▄▄▄ █ █ █▄ █▀▀▄▄█▄█▀▄ █ ███ █
█ ▀▀▀ █ ▄█ ▄▄▄▄█▀▀▀▀█ ▀ ▄▀▄█ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀▄█▄▀ █▄▀ ▀▄█▄█▄▀ █▄▀ ▀▀▀▀▀▀▀
█▀█▀▀ ▀▀▀█▄▀█ ▀ ▄▀▄▀█ ▀▄ █▀▀▄▀ █ █▄█
▄█▄▀ ▀██▀ █▀ ▀▀ ▄█▀ █ █▀▄▀▀▀█▀▄▀
▄█▀▄█▀▀ ▄▀ ▄▄▄█▀▄█▀▄█▄▀▄ ▀▀ ▄▀▀█▀▀█▄▀
▀▀▀██▀█ █ ▀█▀ ▄▄██ ▄▀▄▄█▄█▄▀▀█
▄▄ ▀ ▀ ▀█▀█▄▀▄▀█▀ █▄▄██▄▀█ ▄▀▀ ▀▀█▄▀
▄██▄██▀ ▄▀ ▄█▀ ▀█▀▄ █▄ ▀▀ ██▀██▀ ▀▀█
▄█ ▀ █▀▄█ █▄ ▀▄ ▄▀▄▄▄▄ ▄▀▀▀ ▄█▀▄▀███▀
▀▀█▀▀█▀▀ █▄▄▀ █▄ █▄▀ ▄ █▀█▄▀█ ▀█
██▀▀▄ ▀▀█ ▀█▀▀█▀█▀ █▄▄▀█▀█▄ ▄ ▀▄▀▀██▀
█ █ █▄▀▀▄▄▄▄ ▀█▀█▀█ ▀██▄ ▀█▀█▀ ▀▀ ▀█
▀ ▀▀▀▀▀▀█ ▄▀ ▀ ▄▀▄█▄ █▀ ▀█▄█▀▀▀██▀
█▀▀▀▀▀█ ▀█ ▄▀▀▀▀ ▀ ▄ █ ▄█ █▄█ ▀ █ ▀▀
█ ███ █ █▄▀█▀▀█▀▄█▄▄█▄█▄ ▀█▀▀████▀▀▀▀
█ ▀▀▀ █ █ ▄ ▀ ▀█ ▄ ██ ▀▄▄▄▄▄█ ▀▀█
▀▀▀▀▀▀▀ ▀ ▀ ▀▀ ▀ ▀ ▀▀▀ ▀▀ ▀▀▀▀
Sobol, completed/running/unknown 10/6/4 = ∑20/20, started new job: 0%|░░░░░░░░░░| 0/2000 [03:03, ?it/s]
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.out
Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421 : 0%|░░░░░░░░░░| 1/2000 [03:48<126:40:23, 228.13s/it]
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316661/17316661_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316666/17316666_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316663/17316663_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316664/17316664_0_log.err not found
Sobol, failed: 4, 12 done, failed/running/unknown 10/6/4 = ∑20/20, started new job: 1%|░░░░░░░░░░| 12/2000 [08:05<125:58:34, 228.13s/it]
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.out
Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed : 1%|░░░░░░░░░░| 12/2000 [08:18<125:58:34, 228.13s/it]
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316700/17316700_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316695/17316695_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316694/17316694_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316699/17316699_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316688/17316688_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316691/17316691_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316701/17316701_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316702/17316702_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316687/17316687_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316689/17316689_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316693/17316693_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316698/17316698_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316696/17316696_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316690/17316690_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316692/17316692_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316697/17316697_0_log.err not found
Sobol, failed: 20, 12 done, running/failed 1/3 = ∑4/20, finishing jobs, finished 16 jobs: 1%|░░░░░░░░░░| 12/2000 [09:12<125:58:34, 228.13s/it]
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.out
Sobol, failed: 20, 12 done, running 1 = ∑1/20, job_failed : 1%|░░░░░░░░░░| 12/2000 [09:17<125:58:34, 228.13s/it]
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316713/17316713_0_log.err not found
Sobol, failed: 21, 12 done, failed/running/unknown 15/1/4 = ∑20/20, started new job : 1%|░░░░░░░░░░| 12/2000 [13:55<125:58:34, 228.13s/it]
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.out
⚠ Job (task=0) failed during processing with trace:
----------------------
Traceback (most recent call last):
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/home/pwinkler/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 137, in result
self._result = self.function(*self.args, **self.kwargs)
File "<@beartype(__main__.evaluate) at 0x7ffef78fe4d0>", line 36, in evaluate
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3517, in evaluate
_evaluate_print_stuff(
File "<@beartype(__main__._evaluate_print_stuff) at 0x7ffef78fdea0>", line 269, in _evaluate_print_stuff
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3386, in _evaluate_print_stuff
write_job_infos_csv(parameters, stdout, program_string_with_params, exit_code, _signal, result, start_time, end_time, run_time)
File "<@beartype(__main__.write_job_infos_csv) at 0x7ffef78fd990>", line 230, in write_job_infos_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 3284, in write_job_infos_csv
add_to_csv(f"{get_current_run_folder()}/job_infos.csv", headline, values)
File "<@beartype(__main__.add_to_csv) at 0x7ffef78fc430>", line 68, in add_to_csv
File "/data/horse/ws/pwinkler-oopt/omniopt/.omniopt.py", line 2925, in add_to_csv
rows = list(csv.reader(file))
_csv.Error: line contains NUL
----------------------
You can check full logs with 'job.stderr(0)' and 'job.stdout(0)'or at paths:
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.err
- /data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.out
Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed : 1%|░░░░░░░░░░| 12/2000 [14:10<125:58:34, 228.13s/it]
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316736/17316736_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316733/17316733_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316741/17316741_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316742/17316742_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316739/17316739_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316738/17316738_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316745/17316745_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316743/17316743_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316735/17316735_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316734/17316734_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316746/17316746_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316740/17316740_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316744/17316744_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316747/17316747_0_log.err not found
/data/horse/ws/pwinkler-oopt/omniopt/runs/vergl_botorch_1GB_20w/0/single_runs/17316737/17316737_0_log.err not found
Sobol, failed: 36, 12 done, eval #2/20 start : 1%|░░░░░░░░░░| 12/2000 [16:59<125:58:34, 228.13s/it]
Runtime (end): 20 minutes and 6 seconds, PID: 2824158
Sobol, failed: 36, 12 done, starting new job : 1%|░░░░░░░░░░| 12/2000 [18:24<125:58:34, 228.13s/it]
2025-05-27 13:37:33: SOBOL, Started OmniOpt2 run...
2025-05-27 13:37:39: Sobol, getting new HP set #1/20
2025-05-27 13:37:42: Sobol, getting new HP set #2/20 | ETA: 1s
2025-05-27 13:37:44: Sobol, getting new HP set #3/20 | ETA: 1s
2025-05-27 13:37:46: Sobol, getting new HP set #4/20 | ETA: 1s
2025-05-27 13:37:48: Sobol, getting new HP set #5/20 | ETA: 1s
2025-05-27 13:37:50: Sobol, getting new HP set #6/20 | ETA: 0s
2025-05-27 13:37:52: Sobol, getting new HP set #7/20 | ETA: 0s
2025-05-27 13:37:55: Sobol, getting new HP set #8/20 | ETA: 0s
2025-05-27 13:37:57: Sobol, getting new HP set #9/20 | ETA: 0s
2025-05-27 13:37:59: Sobol, getting new HP set #10/20 | ETA: 0s
2025-05-27 13:38:01: Sobol, getting new HP set #11/20 | ETA: 0s
2025-05-27 13:38:03: Sobol, getting new HP set #12/20 | ETA: 0s
2025-05-27 13:38:06: Sobol, getting new HP set #13/20 | ETA: 0s
2025-05-27 13:38:08: Sobol, getting new HP set #14/20 | ETA: 0s
2025-05-27 13:38:11: Sobol, getting new HP set #15/20 | ETA: 0s
2025-05-27 13:38:13: Sobol, getting new HP set #16/20 | ETA: 0s
2025-05-27 13:38:15: Sobol, getting new HP set #17/20 | ETA: 0s
2025-05-27 13:38:17: Sobol, getting new HP set #18/20 | ETA: 0s
2025-05-27 13:38:20: Sobol, getting new HP set #19/20 | ETA: 0s
2025-05-27 13:38:25: Sobol, getting new HP set #20/20 | ETA: 0s
2025-05-27 13:38:28: Sobol, requested 20 jobs, got 20, 2.474711298942566 s/job
2025-05-27 13:38:34: Sobol, eval #1/20 start
2025-05-27 13:38:39: Sobol, eval #2/20 start
2025-05-27 13:38:41: Sobol, eval #3/20 start
2025-05-27 13:38:44: Sobol, eval #4/20 start
2025-05-27 13:38:46: Sobol, eval #5/20 start
2025-05-27 13:38:48: Sobol, eval #6/20 start
2025-05-27 13:38:50: Sobol, eval #7/20 start
2025-05-27 13:38:52: Sobol, eval #8/20 start
2025-05-27 13:38:54: Sobol, eval #9/20 start
2025-05-27 13:38:57: Sobol, eval #10/20 start
2025-05-27 13:38:58: Sobol, eval #11/20 start
2025-05-27 13:39:00: Sobol, eval #12/20 start
2025-05-27 13:39:02: Sobol, eval #13/20 start
2025-05-27 13:39:05: Sobol, eval #14/20 start
2025-05-27 13:39:07: Sobol, eval #15/20 start
2025-05-27 13:39:09: Sobol, eval #16/20 start
2025-05-27 13:39:11: Sobol, eval #17/20 start
2025-05-27 13:39:14: Sobol, eval #18/20 start
2025-05-27 13:39:16: Sobol, eval #19/20 start
2025-05-27 13:39:18: Sobol, eval #20/20 start
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:21: Sobol, starting new job
2025-05-27 13:39:22: Sobol, starting new job
2025-05-27 13:39:22: Sobol, starting new job
2025-05-27 13:39:22: Sobol, starting new job
2025-05-27 13:39:22: Sobol, starting new job
2025-05-27 13:39:22: Sobol, starting new job
2025-05-27 13:39:45: Sobol, unknown 7 = ∑7/20, started new job
2025-05-27 13:39:45: Sobol, unknown 7 = ∑7/20, started new job
2025-05-27 13:39:46: Sobol, unknown 8 = ∑8/20, started new job
2025-05-27 13:39:46: Sobol, unknown 8 = ∑8/20, started new job
2025-05-27 13:39:46: Sobol, unknown 9 = ∑9/20, started new job
2025-05-27 13:39:46: Sobol, unknown 9 = ∑9/20, started new job
2025-05-27 13:39:47: Sobol, unknown 12 = ∑12/20, started new job
2025-05-27 13:39:48: Sobol, pending 14 = ∑14/20, started new job
2025-05-27 13:39:49: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:49: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:49: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:50: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:50: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:50: Sobol, pending/unknown 15/1 = ∑16/20, started new job
2025-05-27 13:39:51: Sobol, running/pending/unknown 10/5/1 = ∑16/20, started new job
2025-05-27 13:39:51: Sobol, running/pending/unknown 10/5/1 = ∑16/20, started new job
2025-05-27 13:40:16: Sobol, completed/running/pending 2/8/6 = ∑16/20, starting new job
2025-05-27 13:40:16: Sobol, completed/running/pending 2/8/6 = ∑16/20, starting new job
2025-05-27 13:40:16: Sobol, completed/running/pending 2/8/6 = ∑16/20, starting new job
2025-05-27 13:40:16: Sobol, completed/running/pending 2/8/6 = ∑16/20, starting new job
2025-05-27 13:40:36: Sobol, completed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:40:36: Sobol, completed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:40:37: Sobol, completed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:40:37: Sobol, completed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:40:46: Sobol, completed/failed/pending 12/4/4 = ∑20/20, job_failed
2025-05-27 13:40:47: Sobol, completed/failed/pending 12/4/4 = ∑20/20, job_failed
2025-05-27 13:40:47: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:47: Sobol, completed/failed/pending 12/4/4 = ∑20/20, job_failed
2025-05-27 13:40:47: Sobol, completed/failed/pending 12/4/4 = ∑20/20, job_failed
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.25263157894736843
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.21052631578947367
2025-05-27 13:40:48: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.2684210526315789
2025-05-27 13:40:49: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:49: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.38421052631578945
2025-05-27 13:40:49: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:49: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:40:49: Sobol, completed/failed/pending 12/4/4 = ∑20/20, new result: 0.3631578947368421
2025-05-27 13:41:52: Sobol, failed: 4, 12 done, failed 4 = ∑4/20, finishing jobs, finished 16 jobs
2025-05-27 13:41:58: Sobol, failed: 4, 12 done, getting new HP set #1/20
2025-05-27 13:42:01: Sobol, failed: 4, 12 done, getting new HP set #2/20 | ETA: 1s
2025-05-27 13:42:04: Sobol, failed: 4, 12 done, getting new HP set #3/20 | ETA: 1s
2025-05-27 13:42:07: Sobol, failed: 4, 12 done, getting new HP set #4/20 | ETA: 1s
2025-05-27 13:42:10: Sobol, failed: 4, 12 done, getting new HP set #5/20 | ETA: 1s
2025-05-27 13:42:13: Sobol, failed: 4, 12 done, getting new HP set #6/20 | ETA: 0s
2025-05-27 13:42:16: Sobol, failed: 4, 12 done, getting new HP set #7/20 | ETA: 0s
2025-05-27 13:42:19: Sobol, failed: 4, 12 done, getting new HP set #8/20 | ETA: 0s
2025-05-27 13:42:22: Sobol, failed: 4, 12 done, getting new HP set #9/20 | ETA: 0s
2025-05-27 13:42:25: Sobol, failed: 4, 12 done, getting new HP set #10/20 | ETA: 0s
2025-05-27 13:42:28: Sobol, failed: 4, 12 done, getting new HP set #11/20 | ETA: 0s
2025-05-27 13:42:31: Sobol, failed: 4, 12 done, getting new HP set #12/20 | ETA: 0s
2025-05-27 13:42:35: Sobol, failed: 4, 12 done, getting new HP set #13/20 | ETA: 0s
2025-05-27 13:42:38: Sobol, failed: 4, 12 done, getting new HP set #14/20 | ETA: 0s
2025-05-27 13:42:46: Sobol, failed: 4, 12 done, getting new HP set #15/20 | ETA: 0s
2025-05-27 13:42:50: Sobol, failed: 4, 12 done, getting new HP set #16/20 | ETA: 0s
2025-05-27 13:42:53: Sobol, failed: 4, 12 done, getting new HP set #17/20 | ETA: 0s
2025-05-27 13:42:56: Sobol, failed: 4, 12 done, getting new HP set #18/20 | ETA: 0s
2025-05-27 13:42:59: Sobol, failed: 4, 12 done, getting new HP set #19/20 | ETA: 0s
2025-05-27 13:43:02: Sobol, failed: 4, 12 done, getting new HP set #20/20 | ETA: 0s
2025-05-27 13:43:04: Sobol, failed: 4, 12 done, requested 20 jobs, got 20, 3.3557330846786497 s/job
2025-05-27 13:43:08: Sobol, failed: 4, 12 done, eval #1/20 start
2025-05-27 13:43:11: Sobol, failed: 4, 12 done, eval #2/20 start
2025-05-27 13:43:14: Sobol, failed: 4, 12 done, eval #3/20 start
2025-05-27 13:43:17: Sobol, failed: 4, 12 done, eval #4/20 start
2025-05-27 13:43:20: Sobol, failed: 4, 12 done, eval #5/20 start
2025-05-27 13:43:22: Sobol, failed: 4, 12 done, eval #6/20 start
2025-05-27 13:43:25: Sobol, failed: 4, 12 done, eval #7/20 start
2025-05-27 13:43:28: Sobol, failed: 4, 12 done, eval #8/20 start
2025-05-27 13:43:32: Sobol, failed: 4, 12 done, eval #9/20 start
2025-05-27 13:43:35: Sobol, failed: 4, 12 done, eval #10/20 start
2025-05-27 13:43:38: Sobol, failed: 4, 12 done, eval #11/20 start
2025-05-27 13:43:41: Sobol, failed: 4, 12 done, eval #12/20 start
2025-05-27 13:43:44: Sobol, failed: 4, 12 done, eval #13/20 start
2025-05-27 13:43:48: Sobol, failed: 4, 12 done, eval #14/20 start
2025-05-27 13:43:50: Sobol, failed: 4, 12 done, eval #15/20 start
2025-05-27 13:43:53: Sobol, failed: 4, 12 done, eval #16/20 start
2025-05-27 13:43:56: Sobol, failed: 4, 12 done, eval #17/20 start
2025-05-27 13:43:59: Sobol, failed: 4, 12 done, eval #18/20 start
2025-05-27 13:44:02: Sobol, failed: 4, 12 done, eval #19/20 start
2025-05-27 13:44:05: Sobol, failed: 4, 12 done, eval #20/20 start
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:10: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:11: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:11: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:11: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:11: Sobol, failed: 4, 12 done, starting new job
2025-05-27 13:44:45: Sobol, failed: 4, 12 done, pending/unknown 2/5 = ∑7/20, started new job
2025-05-27 13:44:45: Sobol, failed: 4, 12 done, pending/unknown 2/5 = ∑7/20, started new job
2025-05-27 13:44:45: Sobol, failed: 4, 12 done, pending/unknown 2/6 = ∑8/20, started new job
2025-05-27 13:44:45: Sobol, failed: 4, 12 done, pending/unknown 2/6 = ∑8/20, started new job
2025-05-27 13:44:45: Sobol, failed: 4, 12 done, pending/unknown 2/6 = ∑8/20, started new job
2025-05-27 13:44:46: Sobol, failed: 4, 12 done, pending/unknown 2/6 = ∑8/20, started new job
2025-05-27 13:44:47: Sobol, failed: 4, 12 done, pending/unknown 2/12 = ∑14/20, started new job
2025-05-27 13:44:47: Sobol, failed: 4, 12 done, pending/unknown 2/13 = ∑15/20, started new job
2025-05-27 13:44:48: Sobol, failed: 4, 12 done, pending/unknown 2/14 = ∑16/20, started new job
2025-05-27 13:44:48: Sobol, failed: 4, 12 done, pending/unknown 2/14 = ∑16/20, started new job
2025-05-27 13:44:48: Sobol, failed: 4, 12 done, pending/unknown 2/14 = ∑16/20, started new job
2025-05-27 13:44:49: Sobol, failed: 4, 12 done, pending/unknown 2/14 = ∑16/20, started new job
2025-05-27 13:44:49: Sobol, failed: 4, 12 done, pending/unknown 2/14 = ∑16/20, started new job
2025-05-27 13:44:49: Sobol, failed: 4, 12 done, pending 16 = ∑16/20, started new job
2025-05-27 13:44:50: Sobol, failed: 4, 12 done, pending 16 = ∑16/20, started new job
2025-05-27 13:44:50: Sobol, failed: 4, 12 done, pending 16 = ∑16/20, started new job
2025-05-27 13:45:25: Sobol, failed: 4, 12 done, failed/running 9/7 = ∑16/20, starting new job
2025-05-27 13:45:25: Sobol, failed: 4, 12 done, failed/running 9/7 = ∑16/20, starting new job
2025-05-27 13:45:25: Sobol, failed: 4, 12 done, failed/running 9/7 = ∑16/20, starting new job
2025-05-27 13:45:25: Sobol, failed: 4, 12 done, failed/running 9/7 = ∑16/20, starting new job
2025-05-27 13:45:38: Sobol, failed: 4, 12 done, failed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:45:38: Sobol, failed: 4, 12 done, failed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:45:38: Sobol, failed: 4, 12 done, failed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:45:39: Sobol, failed: 4, 12 done, failed/running/unknown 10/6/4 = ∑20/20, started new job
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:51: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:45:53: Sobol, failed: 4, 12 done, failed/pending 16/4 = ∑20/20, job_failed
2025-05-27 13:46:45: Sobol, failed: 20, 12 done, running/failed 1/3 = ∑4/20, finishing jobs, finished 16 jobs
2025-05-27 13:46:50: Sobol, failed: 20, 12 done, running 1 = ∑1/20, job_failed
2025-05-27 13:46:57: Sobol, failed: 21, 12 done, finishing previous jobs (1), finished 1 job
2025-05-27 13:47:03: Sobol, failed: 21, 12 done, getting new HP set #1/20
2025-05-27 13:47:07: Sobol, failed: 21, 12 done, getting new HP set #2/20 | ETA: 1s
2025-05-27 13:47:11: Sobol, failed: 21, 12 done, getting new HP set #3/20 | ETA: 1s
2025-05-27 13:47:14: Sobol, failed: 21, 12 done, getting new HP set #4/20 | ETA: 1s
2025-05-27 13:47:18: Sobol, failed: 21, 12 done, getting new HP set #5/20 | ETA: 0s
2025-05-27 13:47:22: Sobol, failed: 21, 12 done, getting new HP set #6/20 | ETA: 0s
2025-05-27 13:47:26: Sobol, failed: 21, 12 done, getting new HP set #7/20 | ETA: 0s
2025-05-27 13:47:29: Sobol, failed: 21, 12 done, getting new HP set #8/20 | ETA: 0s
2025-05-27 13:47:33: Sobol, failed: 21, 12 done, getting new HP set #9/20 | ETA: 0s
2025-05-27 13:47:37: Sobol, failed: 21, 12 done, getting new HP set #10/20 | ETA: 0s
2025-05-27 13:47:40: Sobol, failed: 21, 12 done, getting new HP set #11/20 | ETA: 0s
2025-05-27 13:47:44: Sobol, failed: 21, 12 done, getting new HP set #12/20 | ETA: 0s
2025-05-27 13:47:48: Sobol, failed: 21, 12 done, getting new HP set #13/20 | ETA: 0s
2025-05-27 13:47:52: Sobol, failed: 21, 12 done, getting new HP set #14/20 | ETA: 0s
2025-05-27 13:47:56: Sobol, failed: 21, 12 done, getting new HP set #15/20 | ETA: 0s
2025-05-27 13:48:00: Sobol, failed: 21, 12 done, getting new HP set #16/20 | ETA: 0s
2025-05-27 13:48:04: Sobol, failed: 21, 12 done, getting new HP set #17/20 | ETA: 0s
2025-05-27 13:48:08: Sobol, failed: 21, 12 done, getting new HP set #18/20 | ETA: 0s
2025-05-27 13:48:11: Sobol, failed: 21, 12 done, getting new HP set #19/20 | ETA: 0s
2025-05-27 13:48:16: Sobol, failed: 21, 12 done, getting new HP set #20/20 | ETA: 0s
2025-05-27 13:48:19: Sobol, failed: 21, 12 done, requested 20 jobs, got 20, 3.8091718316078187 s/job
2025-05-27 13:48:23: Sobol, failed: 21, 12 done, eval #1/20 start
2025-05-27 13:48:26: Sobol, failed: 21, 12 done, eval #2/20 start
2025-05-27 13:48:30: Sobol, failed: 21, 12 done, eval #3/20 start
2025-05-27 13:48:34: Sobol, failed: 21, 12 done, eval #4/20 start
2025-05-27 13:48:37: Sobol, failed: 21, 12 done, eval #5/20 start
2025-05-27 13:48:41: Sobol, failed: 21, 12 done, eval #6/20 start
2025-05-27 13:48:45: Sobol, failed: 21, 12 done, eval #7/20 start
2025-05-27 13:48:49: Sobol, failed: 21, 12 done, eval #8/20 start
2025-05-27 13:48:52: Sobol, failed: 21, 12 done, eval #9/20 start
2025-05-27 13:48:56: Sobol, failed: 21, 12 done, eval #10/20 start
2025-05-27 13:49:01: Sobol, failed: 21, 12 done, eval #11/20 start
2025-05-27 13:49:04: Sobol, failed: 21, 12 done, eval #12/20 start
2025-05-27 13:49:09: Sobol, failed: 21, 12 done, eval #13/20 start
2025-05-27 13:49:12: Sobol, failed: 21, 12 done, eval #14/20 start
2025-05-27 13:49:16: Sobol, failed: 21, 12 done, eval #15/20 start
2025-05-27 13:49:19: Sobol, failed: 21, 12 done, eval #16/20 start
2025-05-27 13:49:23: Sobol, failed: 21, 12 done, eval #17/20 start
2025-05-27 13:49:26: Sobol, failed: 21, 12 done, eval #18/20 start
2025-05-27 13:49:30: Sobol, failed: 21, 12 done, eval #19/20 start
2025-05-27 13:49:34: Sobol, failed: 21, 12 done, eval #20/20 start
2025-05-27 13:49:39: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:39: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:39: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:39: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:40: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:49:41: Sobol, failed: 21, 12 done, starting new job
2025-05-27 13:50:23: Sobol, failed: 21, 12 done, pending/unknown 5/3 = ∑8/20, started new job
2025-05-27 13:50:24: Sobol, failed: 21, 12 done, pending/unknown 5/5 = ∑10/20, started new job
2025-05-27 13:50:24: Sobol, failed: 21, 12 done, pending/unknown 5/5 = ∑10/20, started new job
2025-05-27 13:50:24: Sobol, failed: 21, 12 done, pending/unknown 5/3 = ∑8/20, started new job
2025-05-27 13:50:24: Sobol, failed: 21, 12 done, pending/unknown 5/5 = ∑10/20, started new job
2025-05-27 13:50:24: Sobol, failed: 21, 12 done, pending/unknown 5/5 = ∑10/20, started new job
2025-05-27 13:50:25: Sobol, failed: 21, 12 done, pending/unknown 5/8 = ∑13/20, started new job
2025-05-27 13:50:25: Sobol, failed: 21, 12 done, pending/unknown 5/9 = ∑14/20, started new job
2025-05-27 13:50:26: Sobol, failed: 21, 12 done, pending/unknown 5/11 = ∑16/20, started new job
2025-05-27 13:50:26: Sobol, failed: 21, 12 done, pending/unknown 5/10 = ∑15/20, started new job
2025-05-27 13:50:27: Sobol, failed: 21, 12 done, pending/unknown 5/11 = ∑16/20, started new job
2025-05-27 13:50:27: Sobol, failed: 21, 12 done, pending/unknown 5/11 = ∑16/20, started new job
2025-05-27 13:50:28: Sobol, failed: 21, 12 done, running/pending/unknown 5/9/2 = ∑16/20, started new job
2025-05-27 13:50:28: Sobol, failed: 21, 12 done, running/pending/unknown 5/9/2 = ∑16/20, started new job
2025-05-27 13:50:29: Sobol, failed: 21, 12 done, running/pending/unknown 5/9/2 = ∑16/20, started new job
2025-05-27 13:50:30: Sobol, failed: 21, 12 done, running/pending/unknown 5/9/2 = ∑16/20, started new job
2025-05-27 13:51:13: Sobol, failed: 21, 12 done, failed/running/pending 12/3/1 = ∑16/20, starting new job
2025-05-27 13:51:14: Sobol, failed: 21, 12 done, failed/running/pending 12/3/1 = ∑16/20, starting new job
2025-05-27 13:51:14: Sobol, failed: 21, 12 done, failed/running/pending 12/3/1 = ∑16/20, starting new job
2025-05-27 13:51:14: Sobol, failed: 21, 12 done, failed/running/pending 12/3/1 = ∑16/20, starting new job
2025-05-27 13:51:27: Sobol, failed: 21, 12 done, failed/running/unknown 15/1/3 = ∑19/20, started new job
2025-05-27 13:51:28: Sobol, failed: 21, 12 done, failed/running/unknown 15/1/4 = ∑20/20, started new job
2025-05-27 13:51:28: Sobol, failed: 21, 12 done, failed/running/unknown 15/1/4 = ∑20/20, started new job
2025-05-27 13:51:28: Sobol, failed: 21, 12 done, failed/running/unknown 15/1/4 = ∑20/20, started new job
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:43: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:44: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:51:45: Sobol, failed: 21, 12 done, failed/running/pending 15/1/4 = ∑20/20, job_failed
2025-05-27 13:52:43: Sobol, failed: 36, 12 done, failed 5 = ∑5/20, finishing jobs, finished 15 jobs
2025-05-27 13:52:51: Sobol, failed: 36, 12 done, getting new HP set #1/20
2025-05-27 13:52:57: Sobol, failed: 36, 12 done, getting new HP set #2/20 | ETA: 1s
2025-05-27 13:53:01: Sobol, failed: 36, 12 done, getting new HP set #3/20 | ETA: 1s
2025-05-27 13:53:05: Sobol, failed: 36, 12 done, getting new HP set #4/20 | ETA: 1s
2025-05-27 13:53:10: Sobol, failed: 36, 12 done, getting new HP set #5/20 | ETA: 1s
2025-05-27 13:53:14: Sobol, failed: 36, 12 done, getting new HP set #6/20 | ETA: 0s
2025-05-27 13:53:19: Sobol, failed: 36, 12 done, getting new HP set #7/20 | ETA: 0s
2025-05-27 13:53:23: Sobol, failed: 36, 12 done, getting new HP set #8/20 | ETA: 0s
2025-05-27 13:53:28: Sobol, failed: 36, 12 done, getting new HP set #9/20 | ETA: 0s
2025-05-27 13:53:32: Sobol, failed: 36, 12 done, getting new HP set #10/20 | ETA: 0s
2025-05-27 13:53:36: Sobol, failed: 36, 12 done, getting new HP set #11/20 | ETA: 0s
2025-05-27 13:53:41: Sobol, failed: 36, 12 done, getting new HP set #12/20 | ETA: 0s
2025-05-27 13:53:47: Sobol, failed: 36, 12 done, getting new HP set #13/20 | ETA: 0s
2025-05-27 13:53:51: Sobol, failed: 36, 12 done, getting new HP set #14/20 | ETA: 0s
2025-05-27 13:53:56: Sobol, failed: 36, 12 done, getting new HP set #15/20 | ETA: 0s
2025-05-27 13:54:01: Sobol, failed: 36, 12 done, getting new HP set #16/20 | ETA: 0s
2025-05-27 13:54:06: Sobol, failed: 36, 12 done, getting new HP set #17/20 | ETA: 0s
2025-05-27 13:54:10: Sobol, failed: 36, 12 done, getting new HP set #18/20 | ETA: 0s
2025-05-27 13:54:15: Sobol, failed: 36, 12 done, getting new HP set #19/20 | ETA: 0s
2025-05-27 13:54:19: Sobol, failed: 36, 12 done, getting new HP set #20/20 | ETA: 0s
2025-05-27 13:54:23: Sobol, failed: 36, 12 done, requested 20 jobs, got 20, 4.649256908893586 s/job
2025-05-27 13:54:28: Sobol, failed: 36, 12 done, eval #1/20 start
2025-05-27 13:54:32: Sobol, failed: 36, 12 done, eval #2/20 start
2025-05-27 13:54:36: Sobol, failed: 36, 12 done, eval #3/20 start
2025-05-27 13:54:41: Sobol, failed: 36, 12 done, eval #4/20 start
2025-05-27 13:54:45: Sobol, failed: 36, 12 done, eval #5/20 start
2025-05-27 13:54:49: Sobol, failed: 36, 12 done, eval #6/20 start
2025-05-27 13:54:54: Sobol, failed: 36, 12 done, eval #7/20 start
2025-05-27 13:54:58: Sobol, failed: 36, 12 done, eval #8/20 start
2025-05-27 13:55:03: Sobol, failed: 36, 12 done, eval #9/20 start
2025-05-27 13:55:07: Sobol, failed: 36, 12 done, eval #10/20 start
2025-05-27 13:55:11: Sobol, failed: 36, 12 done, eval #11/20 start
2025-05-27 13:55:16: Sobol, failed: 36, 12 done, eval #12/20 start
2025-05-27 13:55:20: Sobol, failed: 36, 12 done, eval #13/20 start
2025-05-27 13:55:25: Sobol, failed: 36, 12 done, eval #14/20 start
2025-05-27 13:55:30: Sobol, failed: 36, 12 done, eval #15/20 start
2025-05-27 13:55:34: Sobol, failed: 36, 12 done, eval #16/20 start
2025-05-27 13:55:39: Sobol, failed: 36, 12 done, eval #17/20 start
2025-05-27 13:55:43: Sobol, failed: 36, 12 done, eval #18/20 start
2025-05-27 13:55:47: Sobol, failed: 36, 12 done, eval #19/20 start
2025-05-27 13:55:52: Sobol, failed: 36, 12 done, eval #20/20 start
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:57: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job
2025-05-27 13:55:58: Sobol, failed: 36, 12 done, starting new job