2025-04-02 18:42:33: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301577/16301577_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301580/16301580_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301597/16301597_0_log.err not found
⚠ Job 16301584 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 77, in <module>
__import__('pkg_resources.extern.packaging.specifiers')
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 263, in <module>
class LegacySpecifier(_IndividualSpecifier):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 277, in LegacySpecifier
_regex = re.compile(r"^\s*" + _regex_str + r"\s*$", re.VERBOSE | re.IGNORECASE)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 251, in compile
return _compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 303, in _compile
p = sre_compile.compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_compile.py", line 764, in compile
p = sre_parse.parse(p, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 960, in parse
p.state.flags = fix_flags(str, p.state.flags)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_log.err not found
⚠ Job 16301588 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,116) - Starting with JobEnvironment(job_id=16301588, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,130) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 121, in <module>
from pandas.core.reshape.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/reshape/api.py", line 21, in <module>
from pandas.core.reshape.tile import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_log.err not found
⚠ Job 16301592 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,116) - Starting with JobEnvironment(job_id=16301592, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,139) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 119, in <module>
from pandas.core.computation.api import eval
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/api.py", line 2, in <module>
from pandas.core.computation.eval import eval
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/eval.py", line 15, in <module>
from pandas.core.computation.engines import ENGINES
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/engines.py", line 11, in <module>
from pandas.core.computation.align import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/align.py", line 28, in <module>
from pandas.core.computation.common import result_type_many
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 1012, in get_code
File "<frozen importlib._bootstrap_external>", line 672, in _compile_bytecode
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_log.err not found
⚠ Job 16301594 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,107) - Starting with JobEnvironment(job_id=16301594, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,128) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_submitted.pkl
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
File "<frozen importlib._bootstrap_external>", line 147, in _path_stat
FileNotFoundError: Errno 2] No such file or directory: '/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/api/__init__.cpython-310-x86_64-linux-gnu.so'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 138, in <module>
from pandas import api, arrays, errors, io, plotting, tseries
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 945, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
File "<frozen importlib._bootstrap_external>", line 1411, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1563, in find_spec
File "<frozen importlib._bootstrap_external>", line 161, in _path_isfile
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301600/16301600_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301603/16301603_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301606/16301606_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301609/16301609_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301611/16301611_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301614/16301614_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301619/16301619_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301622/16301622_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301625/16301625_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301634/16301634_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301636/16301636_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301647/16301647_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301638/16301638_0_log.err not found
⚠ Job 16301643 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 74, in <module>
from pkg_resources.extern import appdirs
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/extern/__init__.py", line 52, in create_module
return self.load_module(spec.name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/extern/__init__.py", line 37, in load_module
__import__(extant)
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_log.err not found
⚠ Job 16301652 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 77, in <module>
__import__('pkg_resources.extern.packaging.specifiers')
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 13, in <module>
from .utils import canonicalize_version
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_log.err not found
⚠ Job 16301654 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 23, in <module>
from . import _index_tricks_impl
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_index_tricks_impl.py", line 12, in <module>
import numpy.matrixlib as matrixlib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/__init__.py", line 4, in <module>
from . import defmatrix
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/defmatrix.py", line 12, in <module>
from numpy.linalg import matrix_power
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/__init__.py", line 88, in <module>
from . import _linalg
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 41, in <module>
from numpy._typing import NDArray
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_typing/__init__.py", line 32, in <module>
from ._char_codes import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301656/16301656_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301665/16301665_0_log.err not found
Job SlurmJob<job_id=16301616, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16301631, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16301641, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16301616, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16301631, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16301641, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16301616 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/__init__.py", line 100, in <module>
from . import _internal
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/_internal.py", line 140, in <module>
format_re = re.compile(r'(?P<order1><>|=]?)'
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 251, in compile
return _compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 303, in _compile
p = sre_compile.compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_compile.py", line 764, in compile
p = sre_parse.parse(p, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 950, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 443, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 836, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 443, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 664, in _parse
item = subpattern-1:]
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 164, in __getitem__
def __getitem__(self, index):
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_log.err not found
⚠ Job 16301631 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_log.out
----------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 33, in _get_plugins
for entry_point in pkg_resources.iter_entry_points("submitit"):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 641, in <genexpr>
for entry in dist.get_entry_map(group).values()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2853, in get_entry_map
ep_map = self._ep_map = EntryPoint.parse_map(
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2534, in parse_map
for group, lines in data:
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3190, in split_sections
for line in yield_lines(s):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2394, in yield_lines
for ss in strs:
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2764, in _get_metadata
if self.has_metadata(name):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1401, in has_metadata
return self._has(path)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1599, in _has
return os.path.exists(path)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/genericpath.py", line 19, in exists
os.stat(path)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_log.err not found
⚠ Job 16301641 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301641/16301641_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301641/16301641_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 33, in <module>
import email.parser
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301641/16301641_0_log.err not found
Current git-hash: 748be6af2e38a3c47ed5b1c8cc0139d9175bbc80 (version: 5373)
Runtime (end): 5 seconds, PID: 2848388
To cancel, press CTRLc, then run 'scancel 16301507'
Run-UUID: 4031a86f-ce1d-47b5-9415-a4416702e48a
_________________________________________________
/ \
| OmniOpt2 - Optimizing like Schrödinger’s cat—unti |
| l you look, it's both perfect and terrible. |
\ /
=================================================
\
\
\
.--.
|o_o |
|:_/ |
// \ \
(| | )
/'\_ _/`\
\___)=(___/
omniopt --partition=romeo --experiment_name=OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME --result_names ACCURACY=max RUNTIME=min --mem_gb=64 --time=1440 --worker_timeout=60 --max_eval=500 --num_parallel_jobs=30 --revert_to_random_when_seemingly_exhausted --generate_all_jobs_at_once --decimalrounding=12 --gpus=0 --num_random_steps=20 --send_anonymized_usage_stats --run_program=bW9kdWxlIGxvYWQgR0NDY29yZS8xMC4zLjAgUHl0aG9uICYmIHNvdXJjZSAvZGF0YS9ob3JzZS93cy9zNDEyMjQ4NS1jb21wUGVyZkREL2JlbmNobWFyay92ZW52L2Jpbi9hY3RpdmF0ZSAmJiBweXRob24gL2RhdGEvaG9yc2Uvd3MvczQxMjI0ODUtY29tcFBlcmZERC9iZW5jaG1hcmsvZGZraS9iZW5jaG1hcmtkZC9tYWluLnB5IDEgMSAwIFRNREJhbGFuY2VkNXMgMjAwMCBIb2VmZmRpbmdUcmVlQ2xhc3NpZmllciBPQ0REIHJlY2VudF9zYW1wbGVzX3NpemUgJShyZWNlbnRfc2FtcGxlc19zaXplKSBuX3NhbXBsZXMgJShuX3NhbXBsZXMpIHRocmVzaG9sZCAlKHRocmVzaG9sZCkgb3V0bGllcl9kZXRlY3Rvcl9rd2FyZ3MgJShvdXRsaWVyX2RldGVjdG9yX2t3YXJncyk= --cpus_per_task=1 --model=BOTORCH_MODULAR --run_mode=local --live_share --parameter recent_samples_size range 1 5000 int --parameter n_samples range 1 5000 int --parameter threshold range 0.001 0.999 float --parameter outlier_detector_kwargs fixed \{\'nu\':0.5,\'kernel\':\'rbf\',\'gamma\':\'auto\'\}
Run-folder: runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0
Warning: --num_random_steps is smaller than --num_parallel_jobs. It's recommended that --num_parallel_jobs is the same as or a multiple of --num_random_stepsYou have 2 CPUs available for the main process.No CUDA devices found.Generation strategy: Sobol for 30 trials, BoTorch for subsequent trials.
Run-Program: module load GCCcore/10.3.0 Python && source /data/horse/ws/s4122485-compPerfDD/benchmark/venv/bin/activate && python /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/main.py 1 1 0 TMDBalanced5s 2000 HoeffdingTreeClassifier OCDD recent_samples_size %(recent_samples_size) n_samples %(n_samples) threshold %(threshold) outlier_detector_kwargs %(outlier_detector_kwargs)
Experiment parameters:
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃Name ┃Type ┃Lower bound┃Upper bound┃Values ┃Type ┃Log Scale?┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│recent_samples_size │range│1 │5000 ││int │No │
│n_samples │range│1 │5000 ││int │No │
│threshold │range│0.001 │0.999 ││float│No │
│outlier_detector_kwargs│fixed│││\{\'nu\':0.5,\'kernel\':\'rbf\',\'gamma\':\'auto\'\}│││
└─────────────────────────┴───────┴─────────────┴─────────────┴──────────────────────────────────────────────────────┴───────┴────────────┘
Result-Names:
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃Result-Name┃Min or max?┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ACCURACY │ max│
│RUNTIME │ min│
└─────────────┴─────────────┘
See https://imageseg.scads.de/omniax/share?user_id=s4122485&experiment_name=OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME&run_nr=3 for live-results.
█▀▀▀▀▀█ █▀▀ ▀▄█▄ ▀▀▄▀▀█▀▄ ▀█ ▀▀▀ ▀ ▄▀ █▀▀▀▀▀█
█ ███ █ █▀█▄▀█ █▀▀ ██ █▀ ▄▄▀█▀ ▄▀▀ █▄ █ ███ █
█ ▀▀▀ █ ▄▀▄▄█▀ ▀▄█ ▄█▀▀▀█ █▀▀█▀▀▀▀▀▀▀ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀ █▄▀ █ ▀ █▄█ ▀ █ ▀ ▀▄█ █▄█ ▀ ▀▀▀▀▀▀▀
█▀▄ ▀▀▀ ▄█ █▀▄▄▄▄▀▀▀█▀▀▀█ ▄█▄██ ██▀▄▄ ▄█ ▀█▀▀
▄▄▄█▀ ▀▄▀█▀ ▀█▀▀ █ ▄▀▄█ ▄▀▄▄█▀▀█▀▄▄▀▄▀▀▀▀▀▀
██ ███▀█ ▀ █▄▀▄▀▀▀ ██▀█▀ ██ █▄ ▀▄ ▄▄ █ ▄▄█
▄▄▀█▀▄▀▀█ ▄▄█▄ ▄█▄▀▀█▀▄█ ▀▀▄▀ ▄█▄▀▀█▀▄▀▀ ▀
█▄▀▄▀▄▀█ ▀▀██▀█▀ █▀▀▄ ▄█ █▄ ██ ▀█ █▄ ▄█ ▀▄ ▀
██▄▀ █▀████▄▄█ ▄█ █▀█▄▄▄▀▀▄█▄▀▀▀█ ▀█ ▀▄ ▀
▄▀█▀▀▀██▀ ▀▀▀█ █▀▀█▀▀▀█▀▄█▄█▄ ██ ▄█▀▀▀█ ▄▄█
▀██▀█ ▀ █▄███▀█▀██▄ █ ▀ █ ▄ ██▀▄▀▄▄█ ▀ █▄▀█
▀▄ ▄█▀▀▀▀▀ ▀█▄▄▄▄▄▀▀███▀█▀▄▀▄ █ ▀▄ ▀▀▀▀█▀█
█ █▄▄▄▀█▄ ▀▄ █▀▀ █▄ █▀▄ ▀▀ ▀███▀█ ▀ █ █▀ ▀
▄█ ▀▀▀▄██ ██▄██▀▀▀▀ ▀ █▀ █▀▄██ █▄▀▄ █▄▀ █▄
▄ ▀▀▄ ▀ ▀▄▀▀ ▄ ▀ ▀▄▄█▀ ▄▄▀▀▀█▀▄█▄▀▄█▄▄▀█▀▀
▄█▀ █▀▄▄ ▄▄▀▀█▀ ▄▀▀▄▄▄█▀ ▄█ █▄▀█▀▀▄ ▄▀ ▄▄▀
▄▄▄█ ▀ ▄ ▄▀███▄█▀▀ ▄▄▀ ▀▀▀ ▀▀▄▀▄ ▄▀ ▄▄ ▀
▀ ▀▀ ▀ ▄ ▄▄█▄█▄▀▀▀▄█▀▀▀█ ▄█ ▄▄▄▀█▀▄█▀▀▀█▄ ▄
█▀▀▀▀▀█ ▄▄▀▀ ▀█ █▀▄▀█ ▀ █▀▄▄▀█▀ ▄ ▀▀█ ▀ █▄▀█▄
█ ███ █ ▀▀▀▀█▄▄▄▄ █▀██▀▀█ █▀ █▄▄█▄▀ ▀██▀█▄ ▄▄
█ ▀▀▀ █ ▄▀█▄ ▄▄▀ ▀█▄▀█▄▄ ██ ███▄▀▄▄ █▀ ▀▀▀ ▀
▀▀▀▀▀▀▀ ▀ ▀ ▀ ▀▀▀▀▀ ▀ ▀▀ ▀▀ ▀ ▀ ▀▀ ▀
Failed jobs: 4 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/3/20 (87%/30), job_failed: 0%|░░░░░░░░░░| 0/500 04:41<?, ?it/s]
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301577/16301577_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301580/16301580_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301597/16301597_0_log.err not found
⚠ Job 16301584 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 77, in <module>
__import__('pkg_resources.extern.packaging.specifiers')
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 263, in <module>
class LegacySpecifier(_IndividualSpecifier):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 277, in LegacySpecifier
_regex = re.compile(r"^\s*" + _regex_str + r"\s*$", re.VERBOSE | re.IGNORECASE)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 251, in compile
return _compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 303, in _compile
p = sre_compile.compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_compile.py", line 764, in compile
p = sre_parse.parse(p, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 960, in parse
p.state.flags = fix_flags(str, p.state.flags)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301584/16301584_0_log.err not found
⚠ Job 16301588 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,116) - Starting with JobEnvironment(job_id=16301588, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,130) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 121, in <module>
from pandas.core.reshape.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/reshape/api.py", line 21, in <module>
from pandas.core.reshape.tile import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301588/16301588_0_log.err not foundFailed jobs: 9 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 1/20 (70%/30), job_failed: 0%|░░░░░░░░░░| 0/500 05:22<?, ?it/s]
⚠ Job 16301592 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,116) - Starting with JobEnvironment(job_id=16301592, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,139) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 119, in <module>
from pandas.core.computation.api import eval
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/api.py", line 2, in <module>
from pandas.core.computation.eval import eval
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/eval.py", line 15, in <module>
from pandas.core.computation.engines import ENGINES
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/engines.py", line 11, in <module>
from pandas.core.computation.align import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/computation/align.py", line 28, in <module>
from pandas.core.computation.common import result_type_many
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 1012, in get_code
File "<frozen importlib._bootstrap_external>", line 672, in _compile_bytecode
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301592/16301592_0_log.err not found
⚠ Job 16301594 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_log.out
----------------------------------------
submitit INFO (2025-04-02 18:35:34,107) - Starting with JobEnvironment(job_id=16301594, hostname=n1708, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 18:35:34,128) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_submitted.pkl
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
File "<frozen importlib._bootstrap_external>", line 147, in _path_stat
FileNotFoundError: Errno 2] No such file or directory: '/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/api/__init__.cpython-310-x86_64-linux-gnu.so'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 138, in <module>
from pandas import api, arrays, errors, io, plotting, tseries
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 945, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
File "<frozen importlib._bootstrap_external>", line 1411, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1563, in find_spec
File "<frozen importlib._bootstrap_external>", line 161, in _path_isfile
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301594/16301594_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301600/16301600_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301603/16301603_0_log.err not foundFailed jobs: 20 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 6/1/3 (33%/30), job_failed: 0%|░░░░░░░░░░| 0/500 06:54<?, ?it/s]
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301606/16301606_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301609/16301609_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301611/16301611_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301614/16301614_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301619/16301619_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301622/16301622_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301625/16301625_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301634/16301634_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301636/16301636_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301647/16301647_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301638/16301638_0_log.err not found
⚠ Job 16301643 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 74, in <module>
from pkg_resources.extern import appdirs
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/extern/__init__.py", line 52, in create_module
return self.load_module(spec.name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/extern/__init__.py", line 37, in load_module
__import__(extant)
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301643/16301643_0_log.err not foundFailed jobs: 25 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 3/2 (17%/30), waiting for jobs (11 left), finished 6 jobs: 0%|░░░░░░░░░░| 0/500 07:41<?, ?it/s]
⚠ Job 16301652 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 77, in <module>
__import__('pkg_resources.extern.packaging.specifiers')
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 13, in <module>
from .utils import canonicalize_version
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301652/16301652_0_log.err not found
⚠ Job 16301654 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 23, in <module>
from . import _index_tricks_impl
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_index_tricks_impl.py", line 12, in <module>
import numpy.matrixlib as matrixlib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/__init__.py", line 4, in <module>
from . import defmatrix
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/defmatrix.py", line 12, in <module>
from numpy.linalg import matrix_power
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/__init__.py", line 88, in <module>
from . import _linalg
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 41, in <module>
from numpy._typing import NDArray
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_typing/__init__.py", line 32, in <module>
from ._char_codes import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301654/16301654_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301656/16301656_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301665/16301665_0_log.err not foundJob SlurmJob<job_id=16301616, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16301631, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16301641, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16301616, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedFailed jobs: 26 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 2/2 (13%/30), job_failed: 0%|░░░░░░░░░░| 0/500 08:21<?, ?it/s]Job SlurmJob<job_id=16301631, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16301641, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16301616 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/__init__.py", line 100, in <module>
from . import _internal
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/_internal.py", line 140, in <module>
format_re = re.compile(r'(?P<order1><>|=]?)'
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 251, in compile
return _compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 303, in _compile
p = sre_compile.compile(pattern, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_compile.py", line 764, in compile
p = sre_parse.parse(p, flags)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 950, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 443, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 836, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 443, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 664, in _parse
item = subpattern-1:]
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/sre_parse.py", line 164, in __getitem__
def __getitem__(self, index):
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301616/16301616_0_log.err not found
⚠ Job 16301631 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_log.out
----------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 33, in _get_plugins
for entry_point in pkg_resources.iter_entry_points("submitit"):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 641, in <genexpr>
for entry in dist.get_entry_map(group).values()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2853, in get_entry_map
ep_map = self._ep_map = EntryPoint.parse_map(
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2534, in parse_map
for group, lines in data:
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3190, in split_sections
for line in yield_lines(s):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2394, in yield_lines
for ss in strs:
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2764, in _get_metadata
if self.has_metadata(name):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1401, in has_metadata
return self._has(path)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1599, in _has
return os.path.exists(path)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/genericpath.py", line 19, in exists
os.stat(path)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/OCDD_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16301631/16301631_0_log.err not foundFailed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left): 0%|░░░░░░░░░░| 0/500 10:25<?, ?it/s]
2025-04-02 18:32:07: Initializing model, Started OmniOpt2 run...
2025-04-02 18:32:07: Initializing model, finished 0 jobs
2025-04-02 18:32:08: Initializing model, trying to get 30 next steps (current done: 0, max: 500)
2025-04-02 18:32:08: Initializing model, finishing jobs (_get_next_trials), finished 0 jobs
2025-04-02 18:32:08: Initializing model, getting new hyperparameter set #1/30
2025-04-02 18:32:08: SobolGenerator, getting new hyperparameter set #2/30 | ETA: 17s
2025-04-02 18:32:09: SobolGenerator, getting new hyperparameter set #3/30 | ETA: 16s
2025-04-02 18:32:09: SobolGenerator, getting new hyperparameter set #4/30 | ETA: 16s
2025-04-02 18:32:10: SobolGenerator, getting new hyperparameter set #5/30 | ETA: 15s
2025-04-02 18:32:11: SobolGenerator, getting new hyperparameter set #6/30 | ETA: 15s
2025-04-02 18:32:11: SobolGenerator, getting new hyperparameter set #7/30 | ETA: 14s
2025-04-02 18:32:12: SobolGenerator, getting new hyperparameter set #8/30 | ETA: 14s
2025-04-02 18:32:13: SobolGenerator, getting new hyperparameter set #9/30 | ETA: 13s
2025-04-02 18:32:13: SobolGenerator, getting new hyperparameter set #10/30 | ETA: 13s
2025-04-02 18:32:14: SobolGenerator, getting new hyperparameter set #11/30 | ETA: 12s
2025-04-02 18:32:14: SobolGenerator, getting new hyperparameter set #12/30 | ETA: 11s
2025-04-02 18:32:15: SobolGenerator, getting new hyperparameter set #13/30 | ETA: 10s
2025-04-02 18:32:16: SobolGenerator, getting new hyperparameter set #14/30 | ETA: 10s
2025-04-02 18:32:16: SobolGenerator, getting new hyperparameter set #15/30 | ETA: 9s
2025-04-02 18:32:17: SobolGenerator, getting new hyperparameter set #16/30 | ETA: 8s
2025-04-02 18:32:17: SobolGenerator, getting new hyperparameter set #17/30 | ETA: 8s
2025-04-02 18:32:18: SobolGenerator, getting new hyperparameter set #18/30 | ETA: 7s
2025-04-02 18:32:18: SobolGenerator, getting new hyperparameter set #19/30 | ETA: 6s
2025-04-02 18:32:18: SobolGenerator, getting new hyperparameter set #20/30 | ETA: 6s
2025-04-02 18:32:19: SobolGenerator, getting new hyperparameter set #21/30 | ETA: 5s
2025-04-02 18:32:19: SobolGenerator, getting new hyperparameter set #22/30 | ETA: 4s
2025-04-02 18:32:20: SobolGenerator, getting new hyperparameter set #23/30 | ETA: 4s
2025-04-02 18:32:20: SobolGenerator, getting new hyperparameter set #24/30 | ETA: 3s
2025-04-02 18:32:21: SobolGenerator, getting new hyperparameter set #25/30 | ETA: 3s
2025-04-02 18:32:21: SobolGenerator, getting new hyperparameter set #26/30 | ETA: 2s
2025-04-02 18:32:21: SobolGenerator, getting new hyperparameter set #27/30 | ETA: 2s
2025-04-02 18:32:22: SobolGenerator, getting new hyperparameter set #28/30 | ETA: 1s
2025-04-02 18:32:22: SobolGenerator, getting new hyperparameter set #29/30 | ETA: 1s
2025-04-02 18:32:23: SobolGenerator, getting new hyperparameter set #30/30 | ETA: 0s
2025-04-02 18:32:23: SobolGenerator, starting parameter set
2025-04-02 18:32:24: SobolGenerator, starting new job
2025-04-02 18:32:26: SobolGenerator, unknown 1 (3%/30), started new job
2025-04-02 18:32:26: SobolGenerator, unknown 1 (3%/30), starting parameter set
2025-04-02 18:32:26: SobolGenerator, unknown 1 (3%/30), starting new job
2025-04-02 18:32:28: SobolGenerator, unknown 2 (7%/30), started new job
2025-04-02 18:32:28: SobolGenerator, unknown 2 (7%/30), starting parameter set
2025-04-02 18:32:28: SobolGenerator, unknown 2 (7%/30), starting new job
2025-04-02 18:32:31: SobolGenerator, unknown 3 (10%/30), started new job
2025-04-02 18:32:31: SobolGenerator, unknown 3 (10%/30), starting parameter set
2025-04-02 18:32:32: SobolGenerator, pending 3 (10%/30), starting new job
2025-04-02 18:32:35: SobolGenerator, pending/unknown 3/1 (13%/30), started new job
2025-04-02 18:32:35: SobolGenerator, pending/unknown 3/1 (13%/30), starting parameter set
2025-04-02 18:32:36: SobolGenerator, pending 4 (13%/30), starting new job
2025-04-02 18:32:38: SobolGenerator, pending/unknown 4/1 (17%/30), started new job
2025-04-02 18:32:38: SobolGenerator, pending/unknown 4/1 (17%/30), starting parameter set
2025-04-02 18:32:38: SobolGenerator, pending/unknown 4/1 (17%/30), starting new job
2025-04-02 18:32:40: SobolGenerator, pending/unknown 4/2 (20%/30), started new job
2025-04-02 18:32:40: SobolGenerator, pending/unknown 4/2 (20%/30), starting parameter set
2025-04-02 18:32:41: SobolGenerator, pending/unknown 4/2 (20%/30), starting new job
2025-04-02 18:32:43: SobolGenerator, pending/unknown 6/1 (23%/30), started new job
2025-04-02 18:32:43: SobolGenerator, pending/unknown 6/1 (23%/30), starting parameter set
2025-04-02 18:32:43: SobolGenerator, pending/unknown 6/1 (23%/30), starting new job
2025-04-02 18:32:45: SobolGenerator, pending/unknown 6/2 (27%/30), started new job
2025-04-02 18:32:45: SobolGenerator, pending/unknown 6/2 (27%/30), starting parameter set
2025-04-02 18:32:45: SobolGenerator, pending/unknown 6/2 (27%/30), starting new job
2025-04-02 18:32:47: SobolGenerator, pending/unknown 8/1 (30%/30), started new job
2025-04-02 18:32:47: SobolGenerator, pending/unknown 8/1 (30%/30), starting parameter set
2025-04-02 18:32:47: SobolGenerator, pending/unknown 8/1 (30%/30), starting new job
2025-04-02 18:32:50: SobolGenerator, pending/unknown 8/2 (33%/30), started new job
2025-04-02 18:32:50: SobolGenerator, pending/unknown 8/2 (33%/30), starting parameter set
2025-04-02 18:32:51: SobolGenerator, pending/unknown 8/2 (33%/30), starting new job
2025-04-02 18:32:53: SobolGenerator, pending/unknown 10/1 (37%/30), started new job
2025-04-02 18:32:53: SobolGenerator, pending/unknown 10/1 (37%/30), starting parameter set
2025-04-02 18:32:54: SobolGenerator, pending/unknown 10/1 (37%/30), starting new job
2025-04-02 18:32:56: SobolGenerator, pending/unknown 10/2 (40%/30), started new job
2025-04-02 18:32:56: SobolGenerator, pending/unknown 10/2 (40%/30), starting parameter set
2025-04-02 18:32:56: SobolGenerator, pending/unknown 10/2 (40%/30), starting new job
2025-04-02 18:32:58: SobolGenerator, pending/unknown 12/1 (43%/30), started new job
2025-04-02 18:32:58: SobolGenerator, pending/unknown 12/1 (43%/30), starting parameter set
2025-04-02 18:32:59: SobolGenerator, pending/unknown 12/1 (43%/30), starting new job
2025-04-02 18:33:01: SobolGenerator, pending/unknown 12/2 (47%/30), started new job
2025-04-02 18:33:01: SobolGenerator, pending/unknown 12/2 (47%/30), starting parameter set
2025-04-02 18:33:01: SobolGenerator, pending/unknown 12/2 (47%/30), starting new job
2025-04-02 18:33:03: SobolGenerator, pending/unknown 12/3 (50%/30), started new job
2025-04-02 18:33:03: SobolGenerator, pending/unknown 12/3 (50%/30), starting parameter set
2025-04-02 18:33:04: SobolGenerator, pending/unknown 12/3 (50%/30), starting new job
2025-04-02 18:33:06: SobolGenerator, pending/unknown 14/2 (53%/30), started new job
2025-04-02 18:33:06: SobolGenerator, pending/unknown 14/2 (53%/30), starting parameter set
2025-04-02 18:33:06: SobolGenerator, pending/unknown 14/2 (53%/30), starting new job
2025-04-02 18:33:08: SobolGenerator, pending/unknown 14/3 (57%/30), started new job
2025-04-02 18:33:08: SobolGenerator, pending/unknown 14/3 (57%/30), starting parameter set
2025-04-02 18:33:10: SobolGenerator, pending 17 (57%/30), starting new job
2025-04-02 18:33:12: SobolGenerator, pending/unknown 17/1 (60%/30), started new job
2025-04-02 18:33:12: SobolGenerator, pending/unknown 17/1 (60%/30), starting parameter set
2025-04-02 18:33:13: SobolGenerator, pending/unknown 17/1 (60%/30), starting new job
2025-04-02 18:33:15: SobolGenerator, pending/unknown 18/1 (63%/30), started new job
2025-04-02 18:33:15: SobolGenerator, pending/unknown 18/1 (63%/30), starting parameter set
2025-04-02 18:33:16: SobolGenerator, pending/unknown 18/1 (63%/30), starting new job
2025-04-02 18:33:18: SobolGenerator, pending/unknown 18/2 (67%/30), started new job
2025-04-02 18:33:18: SobolGenerator, pending/unknown 18/2 (67%/30), starting parameter set
2025-04-02 18:33:18: SobolGenerator, pending/unknown 18/2 (67%/30), starting new job
2025-04-02 18:33:20: SobolGenerator, pending/unknown 20/1 (70%/30), started new job
2025-04-02 18:33:20: SobolGenerator, pending/unknown 20/1 (70%/30), starting parameter set
2025-04-02 18:33:20: SobolGenerator, pending/unknown 20/1 (70%/30), starting new job
2025-04-02 18:33:23: SobolGenerator, pending/unknown 20/2 (73%/30), started new job
2025-04-02 18:33:23: SobolGenerator, pending/unknown 20/2 (73%/30), starting parameter set
2025-04-02 18:33:24: SobolGenerator, pending/unknown 20/2 (73%/30), starting new job
2025-04-02 18:33:27: SobolGenerator, pending/unknown 22/1 (77%/30), started new job
2025-04-02 18:33:27: SobolGenerator, pending/unknown 22/1 (77%/30), starting parameter set
2025-04-02 18:33:28: SobolGenerator, pending 23 (77%/30), starting new job
2025-04-02 18:33:31: SobolGenerator, pending/unknown 23/1 (80%/30), started new job
2025-04-02 18:33:31: SobolGenerator, pending/unknown 23/1 (80%/30), starting parameter set
2025-04-02 18:33:32: SobolGenerator, pending/unknown 23/1 (80%/30), starting new job
2025-04-02 18:33:34: SobolGenerator, pending/unknown 23/2 (83%/30), started new job
2025-04-02 18:33:34: SobolGenerator, pending/unknown 23/2 (83%/30), starting parameter set
2025-04-02 18:33:34: SobolGenerator, pending/unknown 23/2 (83%/30), starting new job
2025-04-02 18:33:37: SobolGenerator, pending/unknown 25/1 (87%/30), started new job
2025-04-02 18:33:37: SobolGenerator, pending/unknown 25/1 (87%/30), starting parameter set
2025-04-02 18:33:38: SobolGenerator, pending/unknown 25/1 (87%/30), starting new job
2025-04-02 18:33:39: SobolGenerator, pending/unknown 25/2 (90%/30), started new job
2025-04-02 18:33:39: SobolGenerator, pending/unknown 25/2 (90%/30), starting parameter set
2025-04-02 18:33:40: SobolGenerator, pending/unknown 25/2 (90%/30), starting new job
2025-04-02 18:33:42: SobolGenerator, pending/unknown 27/1 (93%/30), started new job
2025-04-02 18:33:42: SobolGenerator, pending/unknown 27/1 (93%/30), starting parameter set
2025-04-02 18:33:42: SobolGenerator, pending/unknown 27/1 (93%/30), starting new job
2025-04-02 18:33:44: SobolGenerator, pending/unknown 27/2 (97%/30), started new job
2025-04-02 18:33:44: SobolGenerator, pending/unknown 27/2 (97%/30), starting parameter set
2025-04-02 18:33:45: SobolGenerator, pending 29 (97%/30), starting new job
2025-04-02 18:33:48: SobolGenerator, pending/unknown 29/1 (100%/30), started new job
2025-04-02 18:34:00: SobolGenerator, pending 30 (100%/30), finishing jobs after starting them, finished 0 jobs
2025-04-02 18:34:00: SobolGenerator, pending 30 (100%/30), got 30, requested 30
2025-04-02 18:34:15: SobolGenerator, pending 30 (100%/30), finishing previous jobs (30), finished 0 jobs
2025-04-02 18:34:16: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 18:34:36: SobolGenerator, pending 30 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 18:34:36: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 18:34:54: SobolGenerator, pending 30 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 18:34:55: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 18:35:00: SobolGenerator, completed/pending 1/29 (100%/30), job_failed
2025-04-02 18:35:15: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 29 (97%/30), waiting for jobs (30 left), finished 1 job
2025-04-02 18:35:16: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 29 (97%/30), waiting for old jobs to finish (29 left)
2025-04-02 18:35:31: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 29 (97%/30), waiting for jobs (29 left), finished 0 jobs
2025-04-02 18:35:31: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 29 (97%/30), waiting for old jobs to finish (29 left)
2025-04-02 18:35:37: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 29 (97%/30), job_failed
2025-04-02 18:35:49: Failed jobs: 2 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 28 (93%/30), waiting for jobs (29 left), finished 1 job
2025-04-02 18:35:50: Failed jobs: 2 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 28 (93%/30), waiting for old jobs to finish (28 left)
2025-04-02 18:35:58: Failed jobs: 2 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 28 (93%/30), job_failed
2025-04-02 18:36:10: Failed jobs: 3 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 27 (90%/30), waiting for jobs (28 left), finished 1 job
2025-04-02 18:36:10: Failed jobs: 3 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 27 (90%/30), waiting for old jobs to finish (27 left)
2025-04-02 18:36:31: Failed jobs: 3 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/3/20 (90%/30), job_failed
2025-04-02 18:36:48: Failed jobs: 4 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/3/20 (87%/30), job_failed
2025-04-02 18:37:05: Failed jobs: 5 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 2/3/20 (83%/30), job_failed
2025-04-02 18:37:22: Failed jobs: 6 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 1/3/20 (80%/30), job_failed
2025-04-02 18:37:25: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 3/20 (77%/30), job_failed
2025-04-02 18:37:27: Failed jobs: 8 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/20 (73%/30), job_failed
2025-04-02 18:37:30: Failed jobs: 9 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 1/20 (70%/30), job_failed
2025-04-02 18:37:32: Failed jobs: 10 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 20 (67%/30), job_failed
2025-04-02 18:37:34: Failed jobs: 11 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 19 (63%/30), job_failed
2025-04-02 18:37:37: Failed jobs: 12 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 18 (60%/30), job_failed
2025-04-02 18:37:41: Failed jobs: 13 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 17 (57%/30), job_failed
2025-04-02 18:37:45: Failed jobs: 14 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 16 (53%/30), job_failed
2025-04-02 18:37:48: Failed jobs: 15 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 15 (50%/30), job_failed
2025-04-02 18:37:52: Failed jobs: 16 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 14 (47%/30), job_failed
2025-04-02 18:37:54: Failed jobs: 17 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 13 (43%/30), job_failed
2025-04-02 18:37:59: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for jobs (27 left), finished 15 jobs
2025-04-02 18:37:59: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for old jobs to finish (12 left)
2025-04-02 18:38:12: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for jobs (12 left), finished 0 jobs
2025-04-02 18:38:12: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for old jobs to finish (12 left)
2025-04-02 18:38:22: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for jobs (12 left), finished 0 jobs
2025-04-02 18:38:22: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for old jobs to finish (12 left)
2025-04-02 18:38:30: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), job_failed
2025-04-02 18:38:37: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for jobs (12 left), finished 1 job
2025-04-02 18:38:38: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for old jobs to finish (11 left)
2025-04-02 18:38:44: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), job_failed
2025-04-02 18:39:02: Failed jobs: 20 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 6/1/3 (33%/30), job_failed
2025-04-02 18:39:19: Failed jobs: 21 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 5/1/3 (30%/30), job_failed
2025-04-02 18:39:37: Failed jobs: 22 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/1/3 (27%/30), job_failed
2025-04-02 18:39:41: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/1/3 (23%/30), job_failed
2025-04-02 18:39:45: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 3/3 (20%/30), job_failed
2025-04-02 18:39:49: Failed jobs: 25 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 3/2 (17%/30), waiting for jobs (11 left), finished 6 jobs
2025-04-02 18:39:51: Failed jobs: 25 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 3/2 (17%/30), waiting for old jobs to finish (5 left)
2025-04-02 18:40:12: Failed jobs: 25 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 3/2 (17%/30), job_failed
2025-04-02 18:40:29: Failed jobs: 26 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 2/2 (13%/30), job_failed
2025-04-02 18:40:47: Failed jobs: 27 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 1/2 (10%/30), job_failed
2025-04-02 18:40:52: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (5 left), finished 3 jobs
2025-04-02 18:40:52: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:40:58: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:40:58: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:04: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:05: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:12: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:12: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:18: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:18: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:24: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:25: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:32: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:32: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:38: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:39: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:45: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:45: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:52: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:52: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:41:59: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:41:59: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:42:06: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:42:06: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:42:13: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:42:13: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:42:20: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:42:20: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:42:26: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:42:27: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)
2025-04-02 18:42:33: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for jobs (2 left), finished 0 jobs
2025-04-02 18:42:33: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 2 (7%/30), waiting for old jobs to finish (2 left)