2025-04-02 19:12:38: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #18/30 | ETA: 8s
Job SlurmJob<job_id=16302430, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302436, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302430, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302436, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302430 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/overrides.py", line 7, in <module>
from numpy._core._multiarray_umath import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302432/16302432_0_log.err not found
⚠ Job 16302436 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 7, in <module>
import argparse
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/argparse.py", line 92, in <module>
from gettext import gettext as _, ngettext
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302448/16302448_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302450/16302450_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302454/16302454_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302461/16302461_0_log.err not found
Job SlurmJob<job_id=16302440, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302444, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302472, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302440, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302444, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302472, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302440 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 33, in <module>
import email.parser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser, BytesFeedParser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/feedparser.py", line 27, in <module>
from email._policybase import compat32
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/_policybase.py", line 7, in <module>
from email import header
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/header.py", line 43, in <module>
''', re.VERBOSE | re.MULTILINE)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/enum.py", line 980, in __or__
def __or__(self, other):
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_log.err not found
⚠ Job 16302444 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_log.out
----------------------------------------
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
File "<frozen importlib._bootstrap_external>", line 147, in _path_stat
FileNotFoundError: Errno 2] No such file or directory: '/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_typing/__init__.cpython-310-x86_64-linux-gnu.so'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 23, in <module>
from . import _index_tricks_impl
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_index_tricks_impl.py", line 12, in <module>
import numpy.matrixlib as matrixlib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/__init__.py", line 4, in <module>
from . import defmatrix
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/defmatrix.py", line 12, in <module>
from numpy.linalg import matrix_power
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/__init__.py", line 88, in <module>
from . import _linalg
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 41, in <module>
from numpy._typing import NDArray
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 945, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
File "<frozen importlib._bootstrap_external>", line 1411, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1563, in find_spec
File "<frozen importlib._bootstrap_external>", line 161, in _path_isfile
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302457/16302457_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302465/16302465_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302468/16302468_0_log.err not found
⚠ Job 16302472 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_log.out
----------------------------------------
submitit INFO (2025-04-02 19:07:14,316) - Starting with JobEnvironment(job_id=16302472, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 19:07:14,320) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 49, in <module>
from pandas.core.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
from pandas.core.groupby import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
from pandas.core.groupby.generic import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1329, in <module>
class DataFrameGroupBy(GroupByDataFrame]):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1330, in DataFrameGroupBy
_agg_examples_doc = dedent(
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/textwrap.py", line 469, in dedent
text = re.sub(r'(?m)^' + margin, '', text)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 209, in sub
return _compile(pattern, flags).sub(repl, string, count)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302475/16302475_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302479/16302479_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302485/16302485_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302490/16302490_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302498/16302498_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302502/16302502_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302509/16302509_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302513/16302513_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302517/16302517_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302523/16302523_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302530/16302530_0_log.err not found
Job SlurmJob<job_id=16302482, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302493, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302533, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302544, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302482, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302493, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302533, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
Job SlurmJob<job_id=16302544, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302482 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 298, in <module>
_core.getlimits._register_known_types()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/getlimits.py", line 190, in _register_known_types
float32_ma = MachArLike(f32,
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/getlimits.py", line 53, in __init__
self.precision = int(-log10(self.eps))
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_log.err not found
⚠ Job 16302493 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 16, in <module>
from . import npyio
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/npyio.py", line 1, in <module>
from ._npyio_impl import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_npyio_impl.py", line 41, in <module>
class BagObj:
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_log.err not found
⚠ Job 16302533 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_log.out
----------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 49, in <module>
from pandas.core.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
from pandas.core.groupby import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
from pandas.core.groupby.generic import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 68, in <module>
from pandas.core.frame import DataFrame
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/frame.py", line 509, in <module>
class DataFrame(NDFrame, OpsMixin):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/frame.py", line 1324, in DataFrame
def to_string(
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/util/_decorators.py", line 295, in decorate
old_sig = inspect.signature(func)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 3247, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 2995, in from_callable
return _signature_from_callable(obj, sigcls=cls,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 2426, in _signature_from_callable
partialmethod = obj._partialmethod
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302538/16302538_0_log.err not found
⚠ Job 16302544 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 33, in <module>
import email.parser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser, BytesFeedParser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/feedparser.py", line 27, in <module>
from email._policybase import compat32
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/_policybase.py", line 7, in <module>
from email import header
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/header.py", line 20, in <module>
from email import charset as _charset
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302548/16302548_0_log.err not found
Current git-hash: 748be6af2e38a3c47ed5b1c8cc0139d9175bbc80 (version: 5373)
Runtime (end): 1 seconds, PID: 3211396
To cancel, press CTRLc, then run 'scancel 16302354'
Run-UUID: 201da39a-ff2b-4c87-a7e5-f7724e1f754a
_____ _____ __ ___
/\ __`\ __/\ __`\ /\ \__ /'___`\
\ \ \/\ \ ___ ___ ___ /\_\ \ \/\ \ _____\ \ ,_\/\_\ /\ \
\ \ \ \ \ /' __` __`\ /' _ `\/\ \ \ \ \ \/\ '__`\ \ \/\/_/// /__
\ \ \_\ \/\ \/\ \/\ \/\ \/\ \ \ \ \ \_\ \ \ \L\ \ \ \_ // /_\ \
\ \_____\ \_\ \_\ \_\ \_\ \_\ \_\ \_____\ \ ,__/\ \__\/\______/
\/_____/\/_/\/_/\/_/\/_/\/_/\/_/\/_____/\ \ \/ \/__/\/_____/
\ \_\
\/_/
omniopt --partition=romeo --experiment_name=BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME --result_names ACCURACY=max RUNTIME=min --mem_gb=32 --time=1440 --worker_timeout=60 --max_eval=500 --num_parallel_jobs=30 --revert_to_random_when_seemingly_exhausted --generate_all_jobs_at_once --decimalrounding=12 --gpus=0 --num_random_steps=20 --send_anonymized_usage_stats --run_program=bW9kdWxlIGxvYWQgR0NDY29yZS8xMC4zLjAgUHl0aG9uICYmIHNvdXJjZSAvZGF0YS9ob3JzZS93cy9zNDEyMjQ4NS1jb21wUGVyZkREL2JlbmNobWFyay92ZW52L2Jpbi9hY3RpdmF0ZSAmJiBweXRob24gL2RhdGEvaG9yc2Uvd3MvczQxMjI0ODUtY29tcFBlcmZERC9iZW5jaG1hcmsvZGZraS9iZW5jaG1hcmtkZC9tYWluLnB5IDEgMSAwIFRNREJhbGFuY2VkNXMgMjAwMCBIb2VmZmRpbmdUcmVlQ2xhc3NpZmllciBCTkRNIHJlY2VudF9zYW1wbGVzX3NpemUgJShyZWNlbnRfc2FtcGxlc19zaXplKSBuX3NhbXBsZXMgJShuX3NhbXBsZXMpIGNvbnN0ICUoY29uc3QpIG1heF9kZXB0aCAlKG1heF9kZXB0aCkgdGhyZXNob2xkICUodGhyZXNob2xkKQ== --cpus_per_task=1 --model=BOTORCH_MODULAR --run_mode=local --live_share --parameter recent_samples_size range 1 5000 int --parameter n_samples range 1 5000 int --parameter const range 0.001 10.0 float --parameter max_depth range 1 20 int --parameter threshold range 0.001 0.999 float
Run-folder: runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0
Warning: --num_random_steps is smaller than --num_parallel_jobs. It's recommended that --num_parallel_jobs is the same as or a multiple of --num_random_stepsYou have 2 CPUs available for the main process.No CUDA devices found.Generation strategy: Sobol for 30 trials, BoTorch for subsequent trials.
Run-Program: module load GCCcore/10.3.0 Python && source /data/horse/ws/s4122485-compPerfDD/benchmark/venv/bin/activate && python /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/main.py 1 1 0 TMDBalanced5s 2000 HoeffdingTreeClassifier BNDM recent_samples_size %(recent_samples_size) n_samples %(n_samples) const %(const) max_depth %(max_depth) threshold %(threshold)
Experiment parameters:
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃Name ┃Type ┃Lower bound┃Upper bound┃Type ┃Log Scale?┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│recent_samples_size│range│1 │5000 │int │No │
│n_samples │range│1 │5000 │int │No │
│const │range│0.001 │10 │float│No │
│max_depth │range│1 │20 │int │No │
│threshold │range│0.001 │0.999 │float│No │
└─────────────────────┴───────┴─────────────┴─────────────┴───────┴────────────┘
Result-Names:
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃Result-Name┃Min or max?┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ACCURACY │ max│
│RUNTIME │ min│
└─────────────┴─────────────┘
See https://imageseg.scads.de/omniax/share?user_id=s4122485&experiment_name=BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME&run_nr=5 for live-results.
█▀▀▀▀▀█ █ ▀ ▄▄█▄ ▀▀▄▀▀█▀▄ ▀█ ▀▀▀ ▀ ▄▀ █▀▀▀▀▀█
█ ███ █ ██ ███ █▀▀ ██ █▀ ▄▄▀█▀ ▄▀▀ █▄ █ ███ █
█ ▀▀▀ █ ▄▄▀▀ ▀ ▀▄█ ▄█▀▀▀█ █▀▀█▀▀▀▀▀▀▀ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀▄▀▄█ █ ▀ █▄█ ▀ █ ▀ ▀▄█ █▄█ ▀ ▀▀▀▀▀▀▀
██▄ ██▀▄▄██▀▄▄▄▄▄▀▀▀█▀▀▀█ ▄█▄██ ██▀▄▄ ▄█ ▀█▀▀
▀▀▀▀██▀▄▄██▄▄█▀▀ █ ▄▀▄█ ▄▀▄▄█▀▀█▀▄▄▀▄▀▀▀▀▀▀
█ ▄▀▄▄▀▄▀▀ ▀▄▄▀▄▀▀▀ ██▀█▀ ██ █▄ ▀▄ ▄▄ █ ▄▄█
▀▄▄▀▄▀▀▄▀▀ █ ▄ ▄█▄▀▀█▀▄█ ▀▀▄▀ ▄█▄▀▀█▀▄▀▀ ▀
██▀█▄▀▀▀█ ▄▀ ▀█▀ █▀▀▄ ▄█ █▄ ██ ▀█ █▄ ▄█ ▀▄ ▀
█ █▄▀▀█ █▄▀▀█ ▄█ █▀█▄▄▄▀▀▄█▄▀▀▀█ ▀█ ▀▄ ▀
▀▄ █▀▀▀█▄█▄█▀▀█ █▀▀█▀▀▀█▀▄█▄█▄ ██ ▄█▀▀▀█ ▄▄█
▀▄█ ▀ ███▀█▀█▀██▄ █ ▀ █ ▄ ██▀▄▀▄▄█ ▀ █▄▀█
█▄█▀▀▀▀█▀▄█▀█▄▄▄▄▄▀▀███▀█▀▄▀▄ █ ▀▄ ▀▀▀▀█▀█
▀▀ █▀▄▀█▄ ▄ █▀▀ █▄ █▀▄ ▀▀ ▀███▀█ ▀ █ █▀ ▀
▀ ▄ ▀▀▀█▄▀██▄██▀▀▀▀ ▀ █▀ █▀▄██ █▄▀▄ █▄▀ █▄
▀█ █▀▀ ▄▄▄ ▄▄ ▀ ▀▄▄█▀ ▄▄▀▀▀█▀▄█▄▀▄█▄▄▀█▀▀
▀ █▀▀▀█▀ ▄▀▀█▀ ▄▀▀▄▄▄█▀ ▄█ █▄▀█▀▀▄ ▄▀ ▄▄▀
▄▄▄█ ▀ ▀ █▀█ ▄▄█▀▀ ▄▄▀ ▀▀▀ ▀▀▄▀▄ ▄▀ ▄▄ ▀
▀ ▀▀ ▀▀▄ █▄█▄█▄▀▀▀▄█▀▀▀█ ▄█ ▄▄▄▀█▀▄█▀▀▀█▄ ▄
█▀▀▀▀▀█ ▄▀▄▀ ▀█ █ ▀█ ▀ █▀▄▄▀█▀ ▄ ▀▀█ ▀ █▄▀█▄
█ ███ █ ▀▄▀▀█▄▄▄▄ █▀██▀▀█ █▀ █▄▄█▄▀ ▀██▀█▄ ▄▄
█ ▀▀▀ █ ▄▀ ▄ ▀█▀ ▀█▄▀█▄▄ ██ ███▄▀▄▄ █▀ ▀▀▀ ▀
▀▀▀▀▀▀▀ ▀ ▀ ▀ ▀▀▀▀▀ ▀ ▀▀ ▀▀ ▀ ▀ ▀▀ ▀
Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/4/16 (77%/30), waiting for jobs (23 left), finished 0 jobs: 0%|░░░░░░░░░░| 0/500 05:13<?, ?it/s]Job SlurmJob<job_id=16302430, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302436, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302430, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302436, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302430 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/overrides.py", line 7, in <module>
from numpy._core._multiarray_umath import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302430/16302430_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302432/16302432_0_log.err not found
⚠ Job 16302436 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 7, in <module>
import argparse
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/argparse.py", line 92, in <module>
from gettext import gettext as _, ngettext
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302436/16302436_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302448/16302448_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302450/16302450_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302454/16302454_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302461/16302461_0_log.err not foundJob SlurmJob<job_id=16302440, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302444, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302472, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302440, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedFailed jobs: 11 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 2/1/16 (63%/30), job_failed: 0%|░░░░░░░░░░| 0/500 05:58<?, ?it/s]Job SlurmJob<job_id=16302444, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302472, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302440 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 33, in <module>
import email.parser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser, BytesFeedParser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/feedparser.py", line 27, in <module>
from email._policybase import compat32
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/_policybase.py", line 7, in <module>
from email import header
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/header.py", line 43, in <module>
''', re.VERBOSE | re.MULTILINE)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/enum.py", line 980, in __or__
def __or__(self, other):
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302440/16302440_0_log.err not found
⚠ Job 16302444 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_log.out
----------------------------------------
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
File "<frozen importlib._bootstrap_external>", line 147, in _path_stat
FileNotFoundError: Errno 2] No such file or directory: '/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_typing/__init__.cpython-310-x86_64-linux-gnu.so'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 23, in <module>
from . import _index_tricks_impl
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_index_tricks_impl.py", line 12, in <module>
import numpy.matrixlib as matrixlib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/__init__.py", line 4, in <module>
from . import defmatrix
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/matrixlib/defmatrix.py", line 12, in <module>
from numpy.linalg import matrix_power
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/__init__.py", line 88, in <module>
from . import _linalg
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 41, in <module>
from numpy._typing import NDArray
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 945, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
File "<frozen importlib._bootstrap_external>", line 1411, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1563, in find_spec
File "<frozen importlib._bootstrap_external>", line 161, in _path_isfile
File "<frozen importlib._bootstrap_external>", line 153, in _path_is_mode_type
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302444/16302444_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302457/16302457_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302465/16302465_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302468/16302468_0_log.err not foundFailed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/1/1 (20%/30), waiting for jobs (6 left), finished 0 jobs: 0%|░░░░░░░░░░| 0/500 08:36<?, ?it/s]
⚠ Job 16302472 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_log.out
----------------------------------------
submitit INFO (2025-04-02 19:07:14,316) - Starting with JobEnvironment(job_id=16302472, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-04-02 19:07:14,320) - Loading pickle: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_submitted.pkl
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 49, in <module>
from pandas.core.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
from pandas.core.groupby import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
from pandas.core.groupby.generic import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1329, in <module>
class DataFrameGroupBy(GroupByDataFrame]):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1330, in DataFrameGroupBy
_agg_examples_doc = dedent(
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/textwrap.py", line 469, in dedent
text = re.sub(r'(?m)^' + margin, '', text)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/re.py", line 209, in sub
return _compile(pattern, flags).sub(repl, string, count)
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302472/16302472_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302475/16302475_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302479/16302479_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302485/16302485_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302490/16302490_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302498/16302498_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302502/16302502_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302509/16302509_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302513/16302513_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302517/16302517_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302523/16302523_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302530/16302530_0_log.err not foundJob SlurmJob<job_id=16302482, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302493, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302533, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302544, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302482, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302493, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedJob SlurmJob<job_id=16302533, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failedFailed jobs: 26 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 2/1/1 (13%/30), job_failed: 0%|░░░░░░░░░░| 0/500 09:36<?, ?it/s]Job SlurmJob<job_id=16302544, task_id=0, state="FAILED">, state not in 'early_stopped', 'abandoned', 'cancelled', 'timeout', 'interrupted'], which would be removed from the job list, or 'completed', 'unknown', 'pending', 'running', 'completing'], which would be ignored: failed
⚠ Job 16302482 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 298, in <module>
_core.getlimits._register_known_types()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/getlimits.py", line 190, in _register_known_types
float32_ma = MachArLike(f32,
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/getlimits.py", line 53, in __init__
self.precision = int(-log10(self.eps))
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302482/16302482_0_log.err not found
⚠ Job 16302493 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 181, in <module>
from . import lib
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/__init__.py", line 16, in <module>
from . import npyio
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/npyio.py", line 1, in <module>
from ._npyio_impl import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/lib/_npyio_impl.py", line 41, in <module>
class BagObj:
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302493/16302493_0_log.err not found
⚠ Job 16302533 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_log.out
----------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 52, in process_job
delayed = utils.DelayedSubmission.load(paths.submitted_pickle)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 153, in load
obj = pickle_load(filepath)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/utils.py", line 232, in pickle_load
return pickle.load(ifile)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 469, in subimport
__import__(name)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/__init__.py", line 49, in <module>
from pandas.core.api import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
from pandas.core.groupby import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
from pandas.core.groupby.generic import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 68, in <module>
from pandas.core.frame import DataFrame
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/frame.py", line 509, in <module>
class DataFrame(NDFrame, OpsMixin):
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/core/frame.py", line 1324, in DataFrame
def to_string(
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pandas/util/_decorators.py", line 295, in decorate
old_sig = inspect.signature(func)
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 3247, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 2995, in from_callable
return _signature_from_callable(obj, sigcls=cls,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/inspect.py", line 2426, in _signature_from_callable
partialmethod = obj._partialmethod
KeyboardInterrupt
Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, finished 0 jobs: 0%|░░░░░░░░░░| 0/500 10:03<?, ?it/s]
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302533/16302533_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302538/16302538_0_log.err not found
⚠ Job 16302544 (task: 0) with path /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_result.pkl
has not produced any output (state: FAILED)
No error stream produced. Look at stdout: /data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_log.out
----------------------------------------
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 37, in process_job
env = job_environment.JobEnvironment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/job_environment.py", line 44, in __new__
return plugins.get_job_environment()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 71, in get_job_environment
envs = get_job_environments()
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 88, in get_job_environments
return {env.name(): env for env in _get_plugins()1]}
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/plugins.py", line 24, in _get_plugins
import pkg_resources
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/pkg_resources/__init__.py", line 33, in <module>
import email.parser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser, BytesFeedParser
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/feedparser.py", line 27, in <module>
from email._policybase import compat32
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/_policybase.py", line 7, in <module>
from email import header
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/email/header.py", line 20, in <module>
from email import charset as _charset
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1074, in get_data
KeyboardInterrupt
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302544/16302544_0_log.err not found
/data/horse/ws/s4122485-compPerfDD/benchmark/dfki/benchmarkdd/OmniOpt/runs/BNDM_TMDBalanced5s_HoeffdingTreeClassifier_ACCURACY-RUNTIME/0/single_runs/16302548/16302548_0_log.err not foundFailed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #18/30 | ETA: 8s: 0%|░░░░░░░░░░| 0/500 10:16<?, ?it/s]
2025-04-02 19:02:22: Initializing model, Started OmniOpt2 run...
2025-04-02 19:02:22: Initializing model, finished 0 jobs
2025-04-02 19:02:22: Initializing model, trying to get 30 next steps (current done: 0, max: 500)
2025-04-02 19:02:22: Initializing model, finishing jobs (_get_next_trials), finished 0 jobs
2025-04-02 19:02:22: Initializing model, getting new hyperparameter set #1/30
2025-04-02 19:02:23: SobolGenerator, getting new hyperparameter set #2/30 | ETA: 17s
2025-04-02 19:02:23: SobolGenerator, getting new hyperparameter set #3/30 | ETA: 16s
2025-04-02 19:02:24: SobolGenerator, getting new hyperparameter set #4/30 | ETA: 16s
2025-04-02 19:02:24: SobolGenerator, getting new hyperparameter set #5/30 | ETA: 15s
2025-04-02 19:02:25: SobolGenerator, getting new hyperparameter set #6/30 | ETA: 15s
2025-04-02 19:02:26: SobolGenerator, getting new hyperparameter set #7/30 | ETA: 14s
2025-04-02 19:02:26: SobolGenerator, getting new hyperparameter set #8/30 | ETA: 14s
2025-04-02 19:02:27: SobolGenerator, getting new hyperparameter set #9/30 | ETA: 13s
2025-04-02 19:02:28: SobolGenerator, getting new hyperparameter set #10/30 | ETA: 13s
2025-04-02 19:02:28: SobolGenerator, getting new hyperparameter set #11/30 | ETA: 12s
2025-04-02 19:02:29: SobolGenerator, getting new hyperparameter set #12/30 | ETA: 11s
2025-04-02 19:02:29: SobolGenerator, getting new hyperparameter set #13/30 | ETA: 10s
2025-04-02 19:02:30: SobolGenerator, getting new hyperparameter set #14/30 | ETA: 10s
2025-04-02 19:02:30: SobolGenerator, getting new hyperparameter set #15/30 | ETA: 9s
2025-04-02 19:02:31: SobolGenerator, getting new hyperparameter set #16/30 | ETA: 8s
2025-04-02 19:02:31: SobolGenerator, getting new hyperparameter set #17/30 | ETA: 8s
2025-04-02 19:02:32: SobolGenerator, getting new hyperparameter set #18/30 | ETA: 7s
2025-04-02 19:02:32: SobolGenerator, getting new hyperparameter set #19/30 | ETA: 6s
2025-04-02 19:02:33: SobolGenerator, getting new hyperparameter set #20/30 | ETA: 6s
2025-04-02 19:02:33: SobolGenerator, getting new hyperparameter set #21/30 | ETA: 5s
2025-04-02 19:02:34: SobolGenerator, getting new hyperparameter set #22/30 | ETA: 4s
2025-04-02 19:02:34: SobolGenerator, getting new hyperparameter set #23/30 | ETA: 4s
2025-04-02 19:02:34: SobolGenerator, getting new hyperparameter set #24/30 | ETA: 3s
2025-04-02 19:02:35: SobolGenerator, getting new hyperparameter set #25/30 | ETA: 3s
2025-04-02 19:02:35: SobolGenerator, getting new hyperparameter set #26/30 | ETA: 2s
2025-04-02 19:02:36: SobolGenerator, getting new hyperparameter set #27/30 | ETA: 2s
2025-04-02 19:02:36: SobolGenerator, getting new hyperparameter set #28/30 | ETA: 1s
2025-04-02 19:02:37: SobolGenerator, getting new hyperparameter set #29/30 | ETA: 1s
2025-04-02 19:02:37: SobolGenerator, getting new hyperparameter set #30/30 | ETA: 0s
2025-04-02 19:02:38: SobolGenerator, starting parameter set
2025-04-02 19:02:38: SobolGenerator, starting new job
2025-04-02 19:02:40: SobolGenerator, unknown 1 (3%/30), started new job
2025-04-02 19:02:40: SobolGenerator, unknown 1 (3%/30), starting parameter set
2025-04-02 19:02:41: SobolGenerator, unknown 1 (3%/30), starting new job
2025-04-02 19:02:43: SobolGenerator, pending/unknown 1/1 (7%/30), started new job
2025-04-02 19:02:43: SobolGenerator, pending/unknown 1/1 (7%/30), starting parameter set
2025-04-02 19:02:44: SobolGenerator, pending/unknown 1/1 (7%/30), starting new job
2025-04-02 19:02:46: SobolGenerator, pending/unknown 1/2 (10%/30), started new job
2025-04-02 19:02:46: SobolGenerator, pending/unknown 1/2 (10%/30), starting parameter set
2025-04-02 19:02:47: SobolGenerator, pending/unknown 2/1 (10%/30), starting new job
2025-04-02 19:02:50: SobolGenerator, pending/unknown 2/2 (13%/30), started new job
2025-04-02 19:02:50: SobolGenerator, pending/unknown 2/2 (13%/30), starting parameter set
2025-04-02 19:02:51: SobolGenerator, pending 4 (13%/30), starting new job
2025-04-02 19:02:53: SobolGenerator, pending/unknown 4/1 (17%/30), started new job
2025-04-02 19:02:53: SobolGenerator, pending/unknown 4/1 (17%/30), starting parameter set
2025-04-02 19:02:53: SobolGenerator, pending/unknown 4/1 (17%/30), starting new job
2025-04-02 19:02:55: SobolGenerator, pending/unknown 4/2 (20%/30), started new job
2025-04-02 19:02:55: SobolGenerator, pending/unknown 4/2 (20%/30), starting parameter set
2025-04-02 19:02:55: SobolGenerator, pending/unknown 4/2 (20%/30), starting new job
2025-04-02 19:02:57: SobolGenerator, pending/unknown 6/1 (23%/30), started new job
2025-04-02 19:02:57: SobolGenerator, pending/unknown 6/1 (23%/30), starting parameter set
2025-04-02 19:02:58: SobolGenerator, pending/unknown 6/1 (23%/30), starting new job
2025-04-02 19:03:00: SobolGenerator, pending/unknown 6/2 (27%/30), started new job
2025-04-02 19:03:00: SobolGenerator, pending/unknown 6/2 (27%/30), starting parameter set
2025-04-02 19:03:00: SobolGenerator, pending/unknown 6/2 (27%/30), starting new job
2025-04-02 19:03:02: SobolGenerator, pending/unknown 8/1 (30%/30), started new job
2025-04-02 19:03:02: SobolGenerator, pending/unknown 8/1 (30%/30), starting parameter set
2025-04-02 19:03:03: SobolGenerator, pending/unknown 8/1 (30%/30), starting new job
2025-04-02 19:03:05: SobolGenerator, pending/unknown 8/2 (33%/30), started new job
2025-04-02 19:03:05: SobolGenerator, pending/unknown 8/2 (33%/30), starting parameter set
2025-04-02 19:03:06: SobolGenerator, pending 10 (33%/30), starting new job
2025-04-02 19:03:08: SobolGenerator, pending/unknown 10/1 (37%/30), started new job
2025-04-02 19:03:08: SobolGenerator, pending/unknown 10/1 (37%/30), starting parameter set
2025-04-02 19:03:08: SobolGenerator, pending/unknown 10/1 (37%/30), starting new job
2025-04-02 19:03:11: SobolGenerator, pending/unknown 10/2 (40%/30), started new job
2025-04-02 19:03:11: SobolGenerator, pending/unknown 10/2 (40%/30), starting parameter set
2025-04-02 19:03:12: SobolGenerator, pending 12 (40%/30), starting new job
2025-04-02 19:03:14: SobolGenerator, pending/unknown 12/1 (43%/30), started new job
2025-04-02 19:03:14: SobolGenerator, pending/unknown 12/1 (43%/30), starting parameter set
2025-04-02 19:03:14: SobolGenerator, pending/unknown 12/1 (43%/30), starting new job
2025-04-02 19:03:16: SobolGenerator, pending/unknown 12/2 (47%/30), started new job
2025-04-02 19:03:16: SobolGenerator, pending/unknown 12/2 (47%/30), starting parameter set
2025-04-02 19:03:17: SobolGenerator, pending/unknown 12/2 (47%/30), starting new job
2025-04-02 19:03:19: SobolGenerator, pending/unknown 13/2 (50%/30), started new job
2025-04-02 19:03:19: SobolGenerator, pending/unknown 13/2 (50%/30), starting parameter set
2025-04-02 19:03:19: SobolGenerator, pending/unknown 13/2 (50%/30), starting new job
2025-04-02 19:03:21: SobolGenerator, pending/unknown 13/3 (53%/30), started new job
2025-04-02 19:03:21: SobolGenerator, pending/unknown 13/3 (53%/30), starting parameter set
2025-04-02 19:03:22: SobolGenerator, pending/unknown 13/3 (53%/30), starting new job
2025-04-02 19:03:24: SobolGenerator, pending/unknown 16/1 (57%/30), started new job
2025-04-02 19:03:24: SobolGenerator, pending/unknown 16/1 (57%/30), starting parameter set
2025-04-02 19:03:25: SobolGenerator, pending/unknown 16/1 (57%/30), starting new job
2025-04-02 19:03:27: SobolGenerator, pending/unknown 16/2 (60%/30), started new job
2025-04-02 19:03:27: SobolGenerator, pending/unknown 16/2 (60%/30), starting parameter set
2025-04-02 19:03:28: SobolGenerator, pending 18 (60%/30), starting new job
2025-04-02 19:03:31: SobolGenerator, pending/unknown 18/1 (63%/30), started new job
2025-04-02 19:03:31: SobolGenerator, pending/unknown 18/1 (63%/30), starting parameter set
2025-04-02 19:03:32: SobolGenerator, pending 19 (63%/30), starting new job
2025-04-02 19:03:33: SobolGenerator, pending/unknown 19/1 (67%/30), started new job
2025-04-02 19:03:33: SobolGenerator, pending/unknown 19/1 (67%/30), starting parameter set
2025-04-02 19:03:34: SobolGenerator, pending/unknown 19/1 (67%/30), starting new job
2025-04-02 19:03:37: SobolGenerator, pending/unknown 19/2 (70%/30), started new job
2025-04-02 19:03:37: SobolGenerator, pending/unknown 19/2 (70%/30), starting parameter set
2025-04-02 19:03:37: SobolGenerator, pending/unknown 19/2 (70%/30), starting new job
2025-04-02 19:03:39: SobolGenerator, pending/unknown 21/1 (73%/30), started new job
2025-04-02 19:03:39: SobolGenerator, pending/unknown 21/1 (73%/30), starting parameter set
2025-04-02 19:03:40: SobolGenerator, pending/unknown 21/1 (73%/30), starting new job
2025-04-02 19:03:42: SobolGenerator, pending/unknown 21/2 (77%/30), started new job
2025-04-02 19:03:42: SobolGenerator, pending/unknown 21/2 (77%/30), starting parameter set
2025-04-02 19:03:42: SobolGenerator, pending/unknown 21/2 (77%/30), starting new job
2025-04-02 19:03:45: SobolGenerator, pending/unknown 23/1 (80%/30), started new job
2025-04-02 19:03:45: SobolGenerator, pending/unknown 23/1 (80%/30), starting parameter set
2025-04-02 19:03:45: SobolGenerator, pending/unknown 23/1 (80%/30), starting new job
2025-04-02 19:03:48: SobolGenerator, pending/unknown 23/2 (83%/30), started new job
2025-04-02 19:03:48: SobolGenerator, pending/unknown 23/2 (83%/30), starting parameter set
2025-04-02 19:03:49: SobolGenerator, pending 25 (83%/30), starting new job
2025-04-02 19:03:51: SobolGenerator, pending/unknown 25/1 (87%/30), started new job
2025-04-02 19:03:51: SobolGenerator, pending/unknown 25/1 (87%/30), starting parameter set
2025-04-02 19:03:52: SobolGenerator, pending/unknown 25/1 (87%/30), starting new job
2025-04-02 19:03:54: SobolGenerator, pending/unknown 26/1 (90%/30), started new job
2025-04-02 19:03:54: SobolGenerator, pending/unknown 26/1 (90%/30), starting parameter set
2025-04-02 19:03:54: SobolGenerator, pending/unknown 26/1 (90%/30), starting new job
2025-04-02 19:03:56: SobolGenerator, pending/unknown 26/2 (93%/30), started new job
2025-04-02 19:03:56: SobolGenerator, pending/unknown 26/2 (93%/30), starting parameter set
2025-04-02 19:03:56: SobolGenerator, pending/unknown 26/2 (93%/30), starting new job
2025-04-02 19:03:58: SobolGenerator, pending/unknown 26/3 (97%/30), started new job
2025-04-02 19:03:58: SobolGenerator, pending/unknown 26/3 (97%/30), starting parameter set
2025-04-02 19:03:59: SobolGenerator, pending/unknown 26/3 (97%/30), starting new job
2025-04-02 19:04:01: SobolGenerator, pending/unknown 28/2 (100%/30), started new job
2025-04-02 19:04:17: SobolGenerator, pending 30 (100%/30), finishing jobs after starting them, finished 0 jobs
2025-04-02 19:04:17: SobolGenerator, pending 30 (100%/30), got 30, requested 30
2025-04-02 19:04:33: SobolGenerator, pending 30 (100%/30), finishing previous jobs (30), finished 0 jobs
2025-04-02 19:04:34: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 19:04:55: SobolGenerator, pending 30 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 19:04:55: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 19:05:15: SobolGenerator, pending 30 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 19:05:15: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 19:05:35: SobolGenerator, pending 30 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 19:05:35: SobolGenerator, pending 30 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 19:05:53: SobolGenerator, failed/running/pending 2/3/25 (100%/30), waiting for jobs (30 left), finished 0 jobs
2025-04-02 19:05:54: SobolGenerator, failed/running/pending 2/3/25 (100%/30), waiting for old jobs to finish (30 left)
2025-04-02 19:06:15: SobolGenerator, failed/running/pending 2/3/25 (100%/30), job_failed
2025-04-02 19:06:18: Failed jobs: 1 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 3/1/25 (97%/30), job_failed
2025-04-02 19:06:37: Failed jobs: 2 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 1/2/25 (93%/30), job_failed
2025-04-02 19:06:42: Failed jobs: 3 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/25 (90%/30), job_failed
2025-04-02 19:06:45: Failed jobs: 4 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/24 (87%/30), job_failed
2025-04-02 19:06:48: Failed jobs: 5 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/23 (83%/30), job_failed
2025-04-02 19:06:59: Failed jobs: 6 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/22 (80%/30), waiting for jobs (30 left), finished 6 jobs
2025-04-02 19:06:59: Failed jobs: 6 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/22 (80%/30), waiting for old jobs to finish (24 left)
2025-04-02 19:07:06: Failed jobs: 6 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/22 (80%/30), job_failed
2025-04-02 19:07:16: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/21 (77%/30), waiting for jobs (24 left), finished 1 job
2025-04-02 19:07:16: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 2/21 (77%/30), waiting for old jobs to finish (23 left)
2025-04-02 19:07:35: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/4/16 (77%/30), waiting for jobs (23 left), finished 0 jobs
2025-04-02 19:07:36: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/4/16 (77%/30), waiting for old jobs to finish (23 left)
2025-04-02 19:07:56: Failed jobs: 7 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/4/16 (77%/30), job_failed
2025-04-02 19:08:13: Failed jobs: 8 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 2/4/16 (73%/30), job_failed
2025-04-02 19:08:16: Failed jobs: 9 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 4/1/16 (70%/30), job_failed
2025-04-02 19:08:18: Failed jobs: 10 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 3/1/16 (67%/30), job_failed
2025-04-02 19:08:20: Failed jobs: 11 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 2/1/16 (63%/30), job_failed
2025-04-02 19:08:37: Failed jobs: 12 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 1/1/16 (60%/30), job_failed
2025-04-02 19:08:40: Failed jobs: 13 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/pending 1/16 (57%/30), job_failed
2025-04-02 19:08:42: Failed jobs: 14 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 16 (53%/30), job_failed
2025-04-02 19:08:45: Failed jobs: 15 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 15 (50%/30), job_failed
2025-04-02 19:08:48: Failed jobs: 16 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 14 (47%/30), job_failed
2025-04-02 19:08:59: Failed jobs: 17 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 13 (43%/30), waiting for jobs (23 left), finished 10 jobs
2025-04-02 19:08:59: Failed jobs: 17 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 13 (43%/30), waiting for old jobs to finish (13 left)
2025-04-02 19:09:06: Failed jobs: 17 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 13 (43%/30), job_failed
2025-04-02 19:09:12: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for jobs (13 left), finished 1 job
2025-04-02 19:09:12: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), waiting for old jobs to finish (12 left)
2025-04-02 19:09:19: Failed jobs: 18 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 12 (40%/30), job_failed
2025-04-02 19:09:27: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for jobs (12 left), finished 1 job
2025-04-02 19:09:27: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for old jobs to finish (11 left)
2025-04-02 19:09:37: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for jobs (11 left), finished 0 jobs
2025-04-02 19:09:37: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), waiting for old jobs to finish (11 left)
2025-04-02 19:09:44: Failed jobs: 19 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 11 (37%/30), job_failed
2025-04-02 19:09:48: Failed jobs: 20 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 10 (33%/30), job_failed
2025-04-02 19:09:50: Failed jobs: 21 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 9 (30%/30), job_failed
2025-04-02 19:09:55: Failed jobs: 22 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 8 (27%/30), waiting for jobs (11 left), finished 3 jobs
2025-04-02 19:09:55: Failed jobs: 22 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 8 (27%/30), waiting for old jobs to finish (8 left)
2025-04-02 19:10:02: Failed jobs: 22 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 8 (27%/30), job_failed
2025-04-02 19:10:09: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for jobs (8 left), finished 1 job
2025-04-02 19:10:09: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for old jobs to finish (7 left)
2025-04-02 19:10:17: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for jobs (7 left), finished 0 jobs
2025-04-02 19:10:17: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for old jobs to finish (7 left)
2025-04-02 19:10:27: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for jobs (7 left), finished 0 jobs
2025-04-02 19:10:27: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for old jobs to finish (7 left)
2025-04-02 19:10:37: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for jobs (7 left), finished 0 jobs
2025-04-02 19:10:37: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), waiting for old jobs to finish (7 left)
2025-04-02 19:10:44: Failed jobs: 23 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 7 (23%/30), job_failed
2025-04-02 19:10:47: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 6 (20%/30), waiting for jobs (7 left), finished 1 job
2025-04-02 19:10:48: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 6 (20%/30), waiting for old jobs to finish (6 left)
2025-04-02 19:10:59: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/1/1 (20%/30), waiting for jobs (6 left), finished 0 jobs
2025-04-02 19:11:01: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/1/1 (20%/30), waiting for old jobs to finish (6 left)
2025-04-02 19:11:21: Failed jobs: 24 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 4/1/1 (20%/30), job_failed
2025-04-02 19:11:40: Failed jobs: 25 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 3/1/1 (17%/30), job_failed
2025-04-02 19:11:58: Failed jobs: 26 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/running/pending 2/1/1 (13%/30), job_failed
2025-04-02 19:12:01: Failed jobs: 27 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, running/failed/pending 1/1/1 (10%/30), job_failed
2025-04-02 19:12:19: Failed jobs: 28 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, failed/pending 1/1 (7%/30), job_failed
2025-04-02 19:12:21: Failed jobs: 29 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, pending 1 (3%/30), job_failed
2025-04-02 19:12:24: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, waiting for jobs (6 left), finished 6 jobs
2025-04-02 19:12:25: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, finished 0 jobs
2025-04-02 19:12:25: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, trying to get 30 next steps (current done: 0, max: 500)
2025-04-02 19:12:25: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, finishing jobs (_get_next_trials), finished 0 jobs
2025-04-02 19:12:26: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #1/30
2025-04-02 19:12:26: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #2/30 | ETA: 18s
2025-04-02 19:12:27: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #3/30 | ETA: 18s
2025-04-02 19:12:28: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #4/30 | ETA: 18s
2025-04-02 19:12:28: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #5/30 | ETA: 17s
2025-04-02 19:12:29: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #6/30 | ETA: 17s
2025-04-02 19:12:30: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #7/30 | ETA: 16s
2025-04-02 19:12:31: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #8/30 | ETA: 15s
2025-04-02 19:12:31: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #9/30 | ETA: 14s
2025-04-02 19:12:32: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #10/30 | ETA: 14s
2025-04-02 19:12:33: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #11/30 | ETA: 13s
2025-04-02 19:12:34: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #12/30 | ETA: 12s
2025-04-02 19:12:34: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #13/30 | ETA: 12s
2025-04-02 19:12:35: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #14/30 | ETA: 11s
2025-04-02 19:12:36: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #15/30 | ETA: 10s
2025-04-02 19:12:37: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #16/30 | ETA: 10s
2025-04-02 19:12:38: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #17/30 | ETA: 9s
2025-04-02 19:12:38: Failed jobs: 30 ('ACCURACY: <number>' not found in output, 'RUNTIME: <number>' not found in output), SobolGenerator, getting new hyperparameter set #18/30 | ETA: 8s
Traceback (most recent call last):
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/software/rapids/r23.04/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/_submit.py", line 7, in <module>
from submitit.core.submission import submitit_main
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/submitit/core/submission.py", line 16, in <module>
import numpy # type: ignore # noqa
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File "/home/s4122485/.omniax_x86_64_Python_3.10.4_barnard/lib/python3.10/site-packages/numpy/_core/overrides.py", line 7, in <module>
from numpy._core._multiarray_umath import (
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
KeyboardInterrupt