⚠ Job 18601964 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,763) - Starting with JobEnvironment(job_id=18601964, hostname=n1585, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,765) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_submitted.pkl
slurmstepd: error: *** JOB 18601964 ON n1585 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601914 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,079) - Starting with JobEnvironment(job_id=18601914, hostname=n1626, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,081) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_submitted.pkl
slurmstepd: error: *** JOB 18601914 ON n1626 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601921 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601921, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_submitted.pkl
slurmstepd: error: *** JOB 18601921 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601919 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601919, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_submitted.pkl
slurmstepd: error: *** JOB 18601919 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601935 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:10,179) - Starting with JobEnvironment(job_id=18601935, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:10,181) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_submitted.pkl
slurmstepd: error: *** JOB 18601935 ON n1588 CANCELLED AT 2025-07-24T15:51:35 DUE TO TIME LIMIT ***
⚠ Job 18601920 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601920, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_submitted.pkl
slurmstepd: error: *** JOB 18601920 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601962 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601962, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_submitted.pkl
slurmstepd: error: *** JOB 18601962 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601956 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601956, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,709) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_submitted.pkl
slurmstepd: error: *** JOB 18601956 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601952 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601952, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,710) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_submitted.pkl
slurmstepd: error: *** JOB 18601952 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601948 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:15,307) - Starting with JobEnvironment(job_id=18601948, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:15,310) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_submitted.pkl
slurmstepd: error: *** JOB 18601948 ON n1587 CANCELLED AT 2025-07-24T15:52:35 DUE TO TIME LIMIT ***
⚠ Job 18601916 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,597) - Starting with JobEnvironment(job_id=18601916, hostname=n1598, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_submitted.pkl
slurmstepd: error: *** JOB 18601916 ON n1598 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601940 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601940, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_submitted.pkl
slurmstepd: error: *** JOB 18601940 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601975 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:54:25,590) - Starting with JobEnvironment(job_id=18601975, hostname=n1577, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:54:25,592) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_submitted.pkl
slurmstepd: error: *** JOB 18601975 ON n1577 CANCELLED AT 2025-07-24T15:54:35 DUE TO TIME LIMIT ***
⚠ Job 18601963 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,678) - Starting with JobEnvironment(job_id=18601963, hostname=n1586, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,680) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_submitted.pkl
slurmstepd: error: *** JOB 18601963 ON n1586 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601932 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:10,179) - Starting with JobEnvironment(job_id=18601932, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:10,181) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_submitted.pkl
slurmstepd: error: *** JOB 18601932 ON n1588 CANCELLED AT 2025-07-24T15:51:35 DUE TO TIME LIMIT ***
⚠ Job 18601966 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,954) - Starting with JobEnvironment(job_id=18601966, hostname=n1577, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_submitted.pkl
slurmstepd: error: *** JOB 18601966 ON n1577 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601938 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,747) - Starting with JobEnvironment(job_id=18601938, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,749) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_submitted.pkl
slurmstepd: error: *** JOB 18601938 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601958 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601958, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_submitted.pkl
slurmstepd: error: *** JOB 18601958 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601917 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,705) - Starting with JobEnvironment(job_id=18601917, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,708) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_submitted.pkl
slurmstepd: error: *** JOB 18601917 ON n1589 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601929 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:37,733) - Starting with JobEnvironment(job_id=18601929, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:37,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_submitted.pkl
slurmstepd: error: *** JOB 18601929 ON n1588 CANCELLED AT 2025-07-24T15:51:05 DUE TO TIME LIMIT ***
⚠ Job 18601953 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601953, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,710) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_submitted.pkl
slurmstepd: error: *** JOB 18601953 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601915 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,823) - Starting with JobEnvironment(job_id=18601915, hostname=n1618, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,825) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_submitted.pkl
slurmstepd: error: *** JOB 18601915 ON n1618 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601939 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,748) - Starting with JobEnvironment(job_id=18601939, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,750) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_submitted.pkl
slurmstepd: error: *** JOB 18601939 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601927 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:37,733) - Starting with JobEnvironment(job_id=18601927, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:37,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_submitted.pkl
slurmstepd: error: *** JOB 18601927 ON n1588 CANCELLED AT 2025-07-24T15:51:05 DUE TO TIME LIMIT ***
⚠ Job 18601937 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601937, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_submitted.pkl
slurmstepd: error: *** JOB 18601937 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601959 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601959, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_submitted.pkl
slurmstepd: error: *** JOB 18601959 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601945 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601945, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_submitted.pkl
slurmstepd: error: *** JOB 18601945 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601960 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601960, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_submitted.pkl
slurmstepd: error: *** JOB 18601960 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601923 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601923, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_submitted.pkl
slurmstepd: error: *** JOB 18601923 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601922 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601922, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_submitted.pkl
slurmstepd: error: *** JOB 18601922 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18603031 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603031, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_submitted.pkl
slurmstepd: error: *** JOB 18603031 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603030 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603030, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_submitted.pkl
slurmstepd: error: *** JOB 18603030 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602999 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,631) - Starting with JobEnvironment(job_id=18602999, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,634) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_submitted.pkl
slurmstepd: error: *** JOB 18602999 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18602991 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602991, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_submitted.pkl
slurmstepd: error: *** JOB 18602991 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603029 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603029, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_submitted.pkl
slurmstepd: error: *** JOB 18603029 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602985 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,898) - Starting with JobEnvironment(job_id=18602985, hostname=n1609, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,900) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_submitted.pkl
slurmstepd: error: *** JOB 18602985 ON n1609 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18602998 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,714) - Starting with JobEnvironment(job_id=18602998, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,717) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_submitted.pkl
slurmstepd: error: *** JOB 18602998 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603066, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,597) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_submitted.pkl
slurmstepd: error: *** JOB 18603066 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603032 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:33,328) - Starting with JobEnvironment(job_id=18603032, hostname=n1439, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:33,330) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_submitted.pkl
slurmstepd: error: *** JOB 18603032 ON n1439 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603131 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603131, hostname=n1397, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,401) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_submitted.pkl
slurmstepd: error: *** JOB 18603131 ON n1397 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18602987 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602987, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,714) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_submitted.pkl
slurmstepd: error: *** JOB 18602987 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603095 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603095, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_submitted.pkl
slurmstepd: error: *** JOB 18603095 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603024 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:33,558) - Starting with JobEnvironment(job_id=18603024, hostname=n1539, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:33,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_submitted.pkl
slurmstepd: error: *** JOB 18603024 ON n1539 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603112 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,368) - Starting with JobEnvironment(job_id=18603112, hostname=n1432, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,370) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_submitted.pkl
slurmstepd: error: *** JOB 18603112 ON n1432 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603051 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:08,011) - Starting with JobEnvironment(job_id=18603051, hostname=n1408, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:08,034) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_submitted.pkl
slurmstepd: error: *** JOB 18603051 ON n1408 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18602995 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,976) - Starting with JobEnvironment(job_id=18602995, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,980) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_submitted.pkl
slurmstepd: error: *** JOB 18602995 ON n1589 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603108 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603108, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,597) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_submitted.pkl
slurmstepd: error: *** JOB 18603108 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603107 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603107, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_submitted.pkl
slurmstepd: error: *** JOB 18603107 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603019 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,514) - Starting with JobEnvironment(job_id=18603019, hostname=n1540, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,516) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_submitted.pkl
slurmstepd: error: *** JOB 18603019 ON n1540 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603130 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603130, hostname=n1397, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,400) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_submitted.pkl
slurmstepd: error: *** JOB 18603130 ON n1397 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603115 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,652) - Starting with JobEnvironment(job_id=18603115, hostname=n1398, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,654) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_submitted.pkl
slurmstepd: error: *** JOB 18603115 ON n1398 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18602993 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602993, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_submitted.pkl
slurmstepd: error: *** JOB 18602993 ON n1606 CANCELLED AT 2025-07-24T18:27:56 DUE TO TIME LIMIT ***
⚠ Job 18602996 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,710) - Starting with JobEnvironment(job_id=18602996, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,712) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_submitted.pkl
slurmstepd: error: *** JOB 18602996 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603109 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603109, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_submitted.pkl
slurmstepd: error: *** JOB 18603109 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18602992 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,712) - Starting with JobEnvironment(job_id=18602992, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,714) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_submitted.pkl
slurmstepd: error: *** JOB 18602992 ON n1606 CANCELLED AT 2025-07-24T18:27:56 DUE TO TIME LIMIT ***
⚠ Job 18603113 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603113, hostname=n1428, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,401) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_submitted.pkl
slurmstepd: error: *** JOB 18603113 ON n1428 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603049 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:08,011) - Starting with JobEnvironment(job_id=18603049, hostname=n1408, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:08,026) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_submitted.pkl
slurmstepd: error: *** JOB 18603049 ON n1408 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603114 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:39,662) - Starting with JobEnvironment(job_id=18603114, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:39,665) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_submitted.pkl
slurmstepd: error: *** JOB 18603114 ON n1399 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603027 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603027, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_submitted.pkl
slurmstepd: error: *** JOB 18603027 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603028 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603028, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_submitted.pkl
slurmstepd: error: *** JOB 18603028 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602990 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602990, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_submitted.pkl
slurmstepd: error: *** JOB 18602990 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603018 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,730) - Starting with JobEnvironment(job_id=18603018, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,733) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_submitted.pkl
slurmstepd: error: *** JOB 18603018 ON n1587 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603869 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:04:24,717) - Starting with JobEnvironment(job_id=18603869, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:04:24,719) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_submitted.pkl
slurmstepd: error: *** JOB 18603869 ON n1707 CANCELLED AT 2025-07-24T21:04:42 DUE TO TIME LIMIT ***
⚠ Job 18603851 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:01:37,133) - Starting with JobEnvironment(job_id=18603851, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:01:37,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_submitted.pkl
slurmstepd: error: *** JOB 18603851 ON n1720 CANCELLED AT 2025-07-24T21:01:42 DUE TO TIME LIMIT ***
⚠ Job 18603858 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603858, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_submitted.pkl
slurmstepd: error: *** JOB 18603858 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603864 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:42,561) - Starting with JobEnvironment(job_id=18603864, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:42,564) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_submitted.pkl
slurmstepd: error: *** JOB 18603864 ON n1719 CANCELLED AT 2025-07-24T21:02:42 DUE TO TIME LIMIT ***
⚠ Job 18603859 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603859, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_submitted.pkl
slurmstepd: error: *** JOB 18603859 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603857 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603857, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_submitted.pkl
slurmstepd: error: *** JOB 18603857 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603862 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603862, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_submitted.pkl
slurmstepd: error: *** JOB 18603862 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603849 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:01:37,133) - Starting with JobEnvironment(job_id=18603849, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:01:37,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_submitted.pkl
slurmstepd: error: *** JOB 18603849 ON n1720 CANCELLED AT 2025-07-24T21:01:42 DUE TO TIME LIMIT ***
⚠ Job 18603867 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:03:54,459) - Starting with JobEnvironment(job_id=18603867, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:03:54,461) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_submitted.pkl
slurmstepd: error: *** JOB 18603867 ON n1707 CANCELLED AT 2025-07-24T21:04:12 DUE TO TIME LIMIT ***
⚠ Job 18603856 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:08,463) - Starting with JobEnvironment(job_id=18603856, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:08,465) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_submitted.pkl
slurmstepd: error: *** JOB 18603856 ON n1720 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603855 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:08,463) - Starting with JobEnvironment(job_id=18603855, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:08,466) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_submitted.pkl
slurmstepd: error: *** JOB 18603855 ON n1720 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603861 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603861, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_submitted.pkl
slurmstepd: error: *** JOB 18603861 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603903 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603903, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_submitted.pkl
slurmstepd: error: *** JOB 18603903 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603915 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:01,516) - Starting with JobEnvironment(job_id=18603915, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:01,518) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_submitted.pkl
slurmstepd: error: *** JOB 18603915 ON n1693 CANCELLED AT 2025-07-24T21:11:12 DUE TO TIME LIMIT ***
⚠ Job 18603899 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:21,987) - Starting with JobEnvironment(job_id=18603899, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:21,989) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_submitted.pkl
slurmstepd: error: *** JOB 18603899 ON n1695 CANCELLED AT 2025-07-24T21:09:42 DUE TO TIME LIMIT ***
⚠ Job 18603921 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:34,494) - Starting with JobEnvironment(job_id=18603921, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:34,496) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_submitted.pkl
slurmstepd: error: *** JOB 18603921 ON n1693 CANCELLED AT 2025-07-24T21:11:42 DUE TO TIME LIMIT ***
⚠ Job 18603887 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:42,364) - Starting with JobEnvironment(job_id=18603887, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:42,366) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_submitted.pkl
slurmstepd: error: *** JOB 18603887 ON n1695 CANCELLED AT 2025-07-24T21:07:42 DUE TO TIME LIMIT ***
⚠ Job 18603878 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603878, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_submitted.pkl
slurmstepd: error: *** JOB 18603878 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603908 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:00,476) - Starting with JobEnvironment(job_id=18603908, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:00,478) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_submitted.pkl
slurmstepd: error: *** JOB 18603908 ON n1694 CANCELLED AT 2025-07-24T21:11:12 DUE TO TIME LIMIT ***
⚠ Job 18603881 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603881, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_submitted.pkl
slurmstepd: error: *** JOB 18603881 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603902 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603902, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_submitted.pkl
slurmstepd: error: *** JOB 18603902 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603892 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:08:15,382) - Starting with JobEnvironment(job_id=18603892, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:08:15,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_submitted.pkl
slurmstepd: error: *** JOB 18603892 ON n1695 CANCELLED AT 2025-07-24T21:08:43 DUE TO TIME LIMIT ***
⚠ Job 18603920 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:34,494) - Starting with JobEnvironment(job_id=18603920, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:34,496) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_submitted.pkl
slurmstepd: error: *** JOB 18603920 ON n1693 CANCELLED AT 2025-07-24T21:11:42 DUE TO TIME LIMIT ***
⚠ Job 18603901 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:54,359) - Starting with JobEnvironment(job_id=18603901, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:54,361) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_submitted.pkl
slurmstepd: error: *** JOB 18603901 ON n1695 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603906 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603906, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_submitted.pkl
slurmstepd: error: *** JOB 18603906 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603893 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:08:15,382) - Starting with JobEnvironment(job_id=18603893, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:08:15,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_submitted.pkl
slurmstepd: error: *** JOB 18603893 ON n1695 CANCELLED AT 2025-07-24T21:08:43 DUE TO TIME LIMIT ***
⚠ Job 18603876 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:03,133) - Starting with JobEnvironment(job_id=18603876, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:03,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_submitted.pkl
slurmstepd: error: *** JOB 18603876 ON n1707 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603886 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:09,087) - Starting with JobEnvironment(job_id=18603886, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:09,089) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_submitted.pkl
slurmstepd: error: *** JOB 18603886 ON n1696 CANCELLED AT 2025-07-24T21:07:13 DUE TO TIME LIMIT ***
⚠ Job 18603889 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:42,364) - Starting with JobEnvironment(job_id=18603889, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:42,366) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_submitted.pkl
slurmstepd: error: *** JOB 18603889 ON n1695 CANCELLED AT 2025-07-24T21:07:42 DUE TO TIME LIMIT ***
⚠ Job 18603877 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603877, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_submitted.pkl
slurmstepd: error: *** JOB 18603877 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603904 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603904, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_submitted.pkl
slurmstepd: error: *** JOB 18603904 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18605465 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_0_log.out
----------------------------------------
submitit INFO (2025-07-24 23:28:50,061) - Starting with JobEnvironment(job_id=18605465, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 23:28:50,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_submitted.pkl
slurmstepd: error: *** JOB 18605465 ON n1719 CANCELLED AT 2025-07-25T01:28:58 DUE TO TIME LIMIT ***
⚠ Job 18606023 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,335) - Starting with JobEnvironment(job_id=18606023, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_submitted.pkl
slurmstepd: error: *** JOB 18606023 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606010 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,932) - Starting with JobEnvironment(job_id=18606010, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,936) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_submitted.pkl
slurmstepd: error: *** JOB 18606010 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606032 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:07:19,074) - Starting with JobEnvironment(job_id=18606032, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:07:19,089) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_submitted.pkl
slurmstepd: error: *** JOB 18606032 ON n1719 CANCELLED AT 2025-07-25T05:07:42 DUE TO TIME LIMIT ***
⚠ Job 18606016 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:03:33,366) - Starting with JobEnvironment(job_id=18606016, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:03:33,370) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_submitted.pkl
slurmstepd: error: *** JOB 18606016 ON n1719 CANCELLED AT 2025-07-25T05:03:42 DUE TO TIME LIMIT ***
⚠ Job 18606022 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606022, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_submitted.pkl
slurmstepd: error: *** JOB 18606022 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606027 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606027, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_submitted.pkl
slurmstepd: error: *** JOB 18606027 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606015 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:03:33,497) - Starting with JobEnvironment(job_id=18606015, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:03:33,499) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_submitted.pkl
slurmstepd: error: *** JOB 18606015 ON n1720 CANCELLED AT 2025-07-25T05:03:42 DUE TO TIME LIMIT ***
⚠ Job 18606028 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,363) - Starting with JobEnvironment(job_id=18606028, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,365) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_submitted.pkl
slurmstepd: error: *** JOB 18606028 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606021 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606021, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_submitted.pkl
slurmstepd: error: *** JOB 18606021 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606003 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:13,938) - Starting with JobEnvironment(job_id=18606003, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:13,941) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_submitted.pkl
slurmstepd: error: *** JOB 18606003 ON n1719 CANCELLED AT 2025-07-25T04:58:42 DUE TO TIME LIMIT ***
⚠ Job 18606002 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:13,938) - Starting with JobEnvironment(job_id=18606002, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:13,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_submitted.pkl
slurmstepd: error: *** JOB 18606002 ON n1719 CANCELLED AT 2025-07-25T04:58:42 DUE TO TIME LIMIT ***
⚠ Job 18606013 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:01:23,231) - Starting with JobEnvironment(job_id=18606013, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:01:23,234) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_submitted.pkl
slurmstepd: error: *** JOB 18606013 ON n1719 CANCELLED AT 2025-07-25T05:01:41 DUE TO TIME LIMIT ***
⚠ Job 18606024 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:48,200) - Starting with JobEnvironment(job_id=18606024, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:48,204) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_submitted.pkl
slurmstepd: error: *** JOB 18606024 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606005 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,313) - Starting with JobEnvironment(job_id=18606005, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,316) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_submitted.pkl
slurmstepd: error: *** JOB 18606005 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606004 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,314) - Starting with JobEnvironment(job_id=18606004, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,316) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_submitted.pkl
slurmstepd: error: *** JOB 18606004 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606029 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,336) - Starting with JobEnvironment(job_id=18606029, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,337) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_submitted.pkl
slurmstepd: error: *** JOB 18606029 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606014 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:01:55,667) - Starting with JobEnvironment(job_id=18606014, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:01:55,669) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_submitted.pkl
slurmstepd: error: *** JOB 18606014 ON n1719 CANCELLED AT 2025-07-25T05:02:11 DUE TO TIME LIMIT ***
⚠ Job 18606020 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,335) - Starting with JobEnvironment(job_id=18606020, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_submitted.pkl
slurmstepd: error: *** JOB 18606020 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:09:57,479) - Starting with JobEnvironment(job_id=18606073, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:09:57,483) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_submitted.pkl
slurmstepd: error: *** JOB 18606073 ON n1719 CANCELLED AT 2025-07-25T05:10:13 DUE TO TIME LIMIT ***
⚠ Job 18606604 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,181) - Starting with JobEnvironment(job_id=18606604, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_submitted.pkl
slurmstepd: error: *** JOB 18606604 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606613 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:42,554) - Starting with JobEnvironment(job_id=18606613, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:42,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_submitted.pkl
slurmstepd: error: *** JOB 18606613 ON n1719 CANCELLED AT 2025-07-25T08:30:49 DUE TO TIME LIMIT ***
⚠ Job 18606610 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:12,681) - Starting with JobEnvironment(job_id=18606610, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:12,683) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_submitted.pkl
slurmstepd: error: *** JOB 18606610 ON n1719 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606612 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:42,559) - Starting with JobEnvironment(job_id=18606612, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:42,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_submitted.pkl
slurmstepd: error: *** JOB 18606612 ON n1719 CANCELLED AT 2025-07-25T08:30:49 DUE TO TIME LIMIT ***
⚠ Job 18606603 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,180) - Starting with JobEnvironment(job_id=18606603, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_submitted.pkl
slurmstepd: error: *** JOB 18606603 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606601 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,181) - Starting with JobEnvironment(job_id=18606601, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_submitted.pkl
slurmstepd: error: *** JOB 18606601 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606634 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:18,944) - Starting with JobEnvironment(job_id=18606634, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:18,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_submitted.pkl
slurmstepd: error: *** JOB 18606634 ON n1706 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606623 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606623, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_submitted.pkl
slurmstepd: error: *** JOB 18606623 ON n1707 CANCELLED AT 2025-07-25T08:38:50 DUE TO TIME LIMIT ***
⚠ Job 18606632 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:19,357) - Starting with JobEnvironment(job_id=18606632, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:19,359) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_submitted.pkl
slurmstepd: error: *** JOB 18606632 ON n1707 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606618 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,875) - Starting with JobEnvironment(job_id=18606618, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,878) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_submitted.pkl
slurmstepd: error: *** JOB 18606618 ON n1720 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606620 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,265) - Starting with JobEnvironment(job_id=18606620, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,266) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_submitted.pkl
slurmstepd: error: *** JOB 18606620 ON n1719 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606624 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606624, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_submitted.pkl
slurmstepd: error: *** JOB 18606624 ON n1707 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606636 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:18,944) - Starting with JobEnvironment(job_id=18606636, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:18,946) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_submitted.pkl
slurmstepd: error: *** JOB 18606636 ON n1706 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606627 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606627, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_submitted.pkl
slurmstepd: error: *** JOB 18606627 ON n1707 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606669 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:55:11,886) - Starting with JobEnvironment(job_id=18606669, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:55:11,890) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_submitted.pkl
slurmstepd: error: *** JOB 18606669 ON n1696 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
⚠ Job 18606679 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,375) - Starting with JobEnvironment(job_id=18606679, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,378) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_submitted.pkl
slurmstepd: error: *** JOB 18606679 ON n1696 CANCELLED AT 2025-07-25T08:52:19 DUE TO TIME LIMIT ***
⚠ Job 18606663 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,950) - Starting with JobEnvironment(job_id=18606663, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_submitted.pkl
slurmstepd: error: *** JOB 18606663 ON n1706 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
⚠ Job 18606667 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,951) - Starting with JobEnvironment(job_id=18606667, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_submitted.pkl
slurmstepd: error: *** JOB 18606667 ON n1706 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
⚠ Job 18606682 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:24,915) - Starting with JobEnvironment(job_id=18606682, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:24,916) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_submitted.pkl
slurmstepd: error: *** JOB 18606682 ON n1696 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606686 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606686, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_submitted.pkl
slurmstepd: error: *** JOB 18606686 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606687 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606687, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_submitted.pkl
slurmstepd: error: *** JOB 18606687 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606685 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606685, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_submitted.pkl
slurmstepd: error: *** JOB 18606685 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18608231 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:55,917) - Starting with JobEnvironment(job_id=18608231, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:55,918) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_submitted.pkl
slurmstepd: error: *** JOB 18608231 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608232 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608232, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_submitted.pkl
slurmstepd: error: *** JOB 18608232 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608230 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608230, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_submitted.pkl
slurmstepd: error: *** JOB 18608230 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608233 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,337) - Starting with JobEnvironment(job_id=18608233, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,340) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_submitted.pkl
slurmstepd: error: *** JOB 18608233 ON n1720 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608227 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608227, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_submitted.pkl
slurmstepd: error: *** JOB 18608227 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608235 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608235, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_submitted.pkl
slurmstepd: error: *** JOB 18608235 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608226 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608226, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_submitted.pkl
slurmstepd: error: *** JOB 18608226 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608234 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,300) - Starting with JobEnvironment(job_id=18608234, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,302) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_submitted.pkl
slurmstepd: error: *** JOB 18608234 ON n1720 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608224 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608224, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_submitted.pkl
slurmstepd: error: *** JOB 18608224 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608228 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608228, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_submitted.pkl
slurmstepd: error: *** JOB 18608228 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608236 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608236, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_submitted.pkl
slurmstepd: error: *** JOB 18608236 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608237 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608237, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_submitted.pkl
slurmstepd: error: *** JOB 18608237 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608223 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608223, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_submitted.pkl
slurmstepd: error: *** JOB 18608223 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608225 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608225, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_submitted.pkl
slurmstepd: error: *** JOB 18608225 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608229 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608229, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_submitted.pkl
slurmstepd: error: *** JOB 18608229 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608395 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,557) - Starting with JobEnvironment(job_id=18608395, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,559) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_submitted.pkl
slurmstepd: error: *** JOB 18608395 ON n1695 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608450 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608450, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_submitted.pkl
slurmstepd: error: *** JOB 18608450 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608380 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:47:55,018) - Starting with JobEnvironment(job_id=18608380, hostname=n1691, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:47:55,019) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_submitted.pkl
slurmstepd: error: *** JOB 18608380 ON n1691 CANCELLED AT 2025-07-25T12:48:24 DUE TO TIME LIMIT ***
⚠ Job 18608443 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608443, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_submitted.pkl
slurmstepd: error: *** JOB 18608443 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608444 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608444, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_submitted.pkl
slurmstepd: error: *** JOB 18608444 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608451 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608451, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_submitted.pkl
slurmstepd: error: *** JOB 18608451 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608448 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:51:02,547) - Starting with JobEnvironment(job_id=18608448, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:51:02,549) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_submitted.pkl
slurmstepd: error: *** JOB 18608448 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608394 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,558) - Starting with JobEnvironment(job_id=18608394, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_submitted.pkl
slurmstepd: error: *** JOB 18608394 ON n1695 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608453 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608453, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_submitted.pkl
slurmstepd: error: *** JOB 18608453 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608464 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:51:25,674) - Starting with JobEnvironment(job_id=18608464, hostname=n1629, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:51:25,676) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_submitted.pkl
slurmstepd: error: *** JOB 18608464 ON n1629 CANCELLED AT 2025-07-25T12:51:54 DUE TO TIME LIMIT ***
⚠ Job 18608452 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608452, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_submitted.pkl
slurmstepd: error: *** JOB 18608452 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608397 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,158) - Starting with JobEnvironment(job_id=18608397, hostname=n1691, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,161) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_submitted.pkl
slurmstepd: error: *** JOB 18608397 ON n1691 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608446 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608446, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_submitted.pkl
slurmstepd: error: *** JOB 18608446 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608525 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:54:55,941) - Starting with JobEnvironment(job_id=18608525, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:54:55,942) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_submitted.pkl
slurmstepd: error: *** JOB 18608525 ON n1687 CANCELLED AT 2025-07-25T12:54:54 DUE TO TIME LIMIT ***
⚠ Job 18608545 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:56:26,046) - Starting with JobEnvironment(job_id=18608545, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:56:26,048) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_submitted.pkl
slurmstepd: error: *** JOB 18608545 ON n1687 CANCELLED AT 2025-07-25T12:56:54 DUE TO TIME LIMIT ***
⚠ Job 18612983 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,908) - Starting with JobEnvironment(job_id=18612983, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,910) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_submitted.pkl
slurmstepd: error: *** JOB 18612983 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612937 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:55:54,431) - Starting with JobEnvironment(job_id=18612937, hostname=n1624, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:55:54,435) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_submitted.pkl
slurmstepd: error: *** JOB 18612937 ON n1624 CANCELLED AT 2025-07-25T16:56:05 DUE TO TIME LIMIT ***
⚠ Job 18612977 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,941) - Starting with JobEnvironment(job_id=18612977, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_submitted.pkl
slurmstepd: error: *** JOB 18612977 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612978 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,938) - Starting with JobEnvironment(job_id=18612978, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_submitted.pkl
slurmstepd: error: *** JOB 18612978 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612981 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,905) - Starting with JobEnvironment(job_id=18612981, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,907) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_submitted.pkl
slurmstepd: error: *** JOB 18612981 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612979 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,937) - Starting with JobEnvironment(job_id=18612979, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,939) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_submitted.pkl
slurmstepd: error: *** JOB 18612979 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612963 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:54,720) - Starting with JobEnvironment(job_id=18612963, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:54,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_submitted.pkl
slurmstepd: error: *** JOB 18612963 ON n1570 CANCELLED AT 2025-07-25T16:57:06 DUE TO TIME LIMIT ***
⚠ Job 18612982 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,941) - Starting with JobEnvironment(job_id=18612982, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_submitted.pkl
slurmstepd: error: *** JOB 18612982 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612962 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:54,720) - Starting with JobEnvironment(job_id=18612962, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:54,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_submitted.pkl
slurmstepd: error: *** JOB 18612962 ON n1570 CANCELLED AT 2025-07-25T16:57:06 DUE TO TIME LIMIT ***
⚠ Job 18612976 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,828) - Starting with JobEnvironment(job_id=18612976, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,831) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_submitted.pkl
slurmstepd: error: *** JOB 18612976 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612950 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:24,010) - Starting with JobEnvironment(job_id=18612950, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:24,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_submitted.pkl
slurmstepd: error: *** JOB 18612950 ON n1589 CANCELLED AT 2025-07-25T16:56:35 DUE TO TIME LIMIT ***
⚠ Job 18612864 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:53:52,971) - Starting with JobEnvironment(job_id=18612864, hostname=n1700, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:53:52,973) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_submitted.pkl
slurmstepd: error: *** JOB 18612864 ON n1700 CANCELLED AT 2025-07-25T16:54:05 DUE TO TIME LIMIT ***
⚠ Job 18613048 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_0_log.out
----------------------------------------
submitit INFO (2025-07-25 15:02:56,501) - Starting with JobEnvironment(job_id=18613048, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 15:02:56,504) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_submitted.pkl
slurmstepd: error: *** JOB 18613048 ON n1719 CANCELLED AT 2025-07-25T17:03:07 DUE TO TIME LIMIT ***
⚠ Job 18613052 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_0_log.out
----------------------------------------
submitit INFO (2025-07-25 15:03:27,938) - Starting with JobEnvironment(job_id=18613052, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 15:03:27,941) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_submitted.pkl
slurmstepd: error: *** JOB 18613052 ON n1719 CANCELLED AT 2025-07-25T17:03:38 DUE TO TIME LIMIT ***
⚠ Job 18612985 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:55,193) - Starting with JobEnvironment(job_id=18612985, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:55,195) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_submitted.pkl
slurmstepd: error: *** JOB 18612985 ON n1570 CANCELLED AT 2025-07-25T16:58:05 DUE TO TIME LIMIT ***
⚠ Job 18614239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614239, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_submitted.pkl
slurmstepd: error: *** JOB 18614239 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614243, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_submitted.pkl
slurmstepd: error: *** JOB 18614243 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,376) - Starting with JobEnvironment(job_id=18614252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_submitted.pkl
slurmstepd: error: *** JOB 18614252 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:00:28,118) - Starting with JobEnvironment(job_id=18614245, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:00:28,121) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_submitted.pkl
slurmstepd: error: *** JOB 18614245 ON n1718 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_submitted.pkl
slurmstepd: error: *** JOB 18614250 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_submitted.pkl
slurmstepd: error: *** JOB 18614248 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614253 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614253, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:34,016) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_submitted.pkl
slurmstepd: error: *** JOB 18614253 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614244, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_submitted.pkl
slurmstepd: error: *** JOB 18614244 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_submitted.pkl
slurmstepd: error: *** JOB 18614251 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614240, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_submitted.pkl
slurmstepd: error: *** JOB 18614240 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614238 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614238, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,037) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_submitted.pkl
slurmstepd: error: *** JOB 18614238 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614241, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_submitted.pkl
slurmstepd: error: *** JOB 18614241 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_submitted.pkl
slurmstepd: error: *** JOB 18614249 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614242, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,037) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_submitted.pkl
slurmstepd: error: *** JOB 18614242 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_submitted.pkl
slurmstepd: error: *** JOB 18614247 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_submitted.pkl
slurmstepd: error: *** JOB 18614246 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614462 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:15:50,380) - Starting with JobEnvironment(job_id=18614462, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:15:50,382) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_submitted.pkl
slurmstepd: error: *** JOB 18614462 ON n1718 CANCELLED AT 2025-07-25T21:16:15 DUE TO TIME LIMIT ***
⚠ Job 18614463 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:21,556) - Starting with JobEnvironment(job_id=18614463, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:21,576) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_submitted.pkl
slurmstepd: error: *** JOB 18614463 ON n1718 CANCELLED AT 2025-07-25T21:16:45 DUE TO TIME LIMIT ***
⚠ Job 18614461 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:15:50,349) - Starting with JobEnvironment(job_id=18614461, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:15:50,365) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_submitted.pkl
slurmstepd: error: *** JOB 18614461 ON n1718 CANCELLED AT 2025-07-25T21:16:15 DUE TO TIME LIMIT ***
⚠ Job 18614474 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614474, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_submitted.pkl
slurmstepd: error: *** JOB 18614474 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614472 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614472, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_submitted.pkl
slurmstepd: error: *** JOB 18614472 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614489 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:23:56,341) - Starting with JobEnvironment(job_id=18614489, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:23:56,344) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_submitted.pkl
slurmstepd: error: *** JOB 18614489 ON n1716 CANCELLED AT 2025-07-25T21:24:15 DUE TO TIME LIMIT ***
⚠ Job 18614470 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,493) - Starting with JobEnvironment(job_id=18614470, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,495) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_submitted.pkl
slurmstepd: error: *** JOB 18614470 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614467 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:51,734) - Starting with JobEnvironment(job_id=18614467, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:51,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_submitted.pkl
slurmstepd: error: *** JOB 18614467 ON n1717 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18614466 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:50,973) - Starting with JobEnvironment(job_id=18614466, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:50,975) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_submitted.pkl
slurmstepd: error: *** JOB 18614466 ON n1718 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18614468 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,346) - Starting with JobEnvironment(job_id=18614468, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,348) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_submitted.pkl
slurmstepd: error: *** JOB 18614468 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614490 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:23:56,341) - Starting with JobEnvironment(job_id=18614490, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:23:56,344) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_submitted.pkl
slurmstepd: error: *** JOB 18614490 ON n1716 CANCELLED AT 2025-07-25T21:24:15 DUE TO TIME LIMIT ***
⚠ Job 18614475 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614475, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_submitted.pkl
slurmstepd: error: *** JOB 18614475 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614473 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614473, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_submitted.pkl
slurmstepd: error: *** JOB 18614473 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614471 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614471, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_submitted.pkl
slurmstepd: error: *** JOB 18614471 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614469 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,346) - Starting with JobEnvironment(job_id=18614469, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,348) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_submitted.pkl
slurmstepd: error: *** JOB 18614469 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614465 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:50,973) - Starting with JobEnvironment(job_id=18614465, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:50,975) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_submitted.pkl
slurmstepd: error: *** JOB 18614465 ON n1718 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18615243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,852) - Starting with JobEnvironment(job_id=18615243, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,854) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_submitted.pkl
slurmstepd: error: *** JOB 18615243 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,896) - Starting with JobEnvironment(job_id=18615249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,898) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_submitted.pkl
slurmstepd: error: *** JOB 18615249 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,903) - Starting with JobEnvironment(job_id=18615248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,905) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_submitted.pkl
slurmstepd: error: *** JOB 18615248 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,852) - Starting with JobEnvironment(job_id=18615242, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,854) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_submitted.pkl
slurmstepd: error: *** JOB 18615242 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615254 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,758) - Starting with JobEnvironment(job_id=18615254, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_submitted.pkl
slurmstepd: error: *** JOB 18615254 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,976) - Starting with JobEnvironment(job_id=18615250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_submitted.pkl
slurmstepd: error: *** JOB 18615250 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:53,791) - Starting with JobEnvironment(job_id=18615245, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:53,793) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_submitted.pkl
slurmstepd: error: *** JOB 18615245 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:22,352) - Starting with JobEnvironment(job_id=18615240, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:22,355) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_submitted.pkl
slurmstepd: error: *** JOB 18615240 ON n1720 CANCELLED AT 2025-07-26T01:19:27 DUE TO TIME LIMIT ***
⚠ Job 18615253 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,757) - Starting with JobEnvironment(job_id=18615253, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_submitted.pkl
slurmstepd: error: *** JOB 18615253 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,864) - Starting with JobEnvironment(job_id=18615241, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,867) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_submitted.pkl
slurmstepd: error: *** JOB 18615241 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,896) - Starting with JobEnvironment(job_id=18615246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,899) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_submitted.pkl
slurmstepd: error: *** JOB 18615246 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,757) - Starting with JobEnvironment(job_id=18615251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_submitted.pkl
slurmstepd: error: *** JOB 18615251 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,976) - Starting with JobEnvironment(job_id=18615247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_submitted.pkl
slurmstepd: error: *** JOB 18615247 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:22,352) - Starting with JobEnvironment(job_id=18615239, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:22,355) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_submitted.pkl
slurmstepd: error: *** JOB 18615239 ON n1720 CANCELLED AT 2025-07-26T01:19:27 DUE TO TIME LIMIT ***
⚠ Job 18615244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:53,790) - Starting with JobEnvironment(job_id=18615244, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:53,793) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_submitted.pkl
slurmstepd: error: *** JOB 18615244 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,758) - Starting with JobEnvironment(job_id=18615252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_submitted.pkl
slurmstepd: error: *** JOB 18615252 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:50:35,268) - Starting with JobEnvironment(job_id=18615319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:50:35,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_submitted.pkl
slurmstepd: error: *** JOB 18615319 ON n1720 CANCELLED AT 2025-07-26T01:50:58 DUE TO TIME LIMIT ***
⚠ Job 18615322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:51:05,763) - Starting with JobEnvironment(job_id=18615322, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:51:05,765) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_submitted.pkl
slurmstepd: error: *** JOB 18615322 ON n1719 CANCELLED AT 2025-07-26T01:51:27 DUE TO TIME LIMIT ***
⚠ Job 18615325 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:52:07,037) - Starting with JobEnvironment(job_id=18615325, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:52:07,041) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_submitted.pkl
slurmstepd: error: *** JOB 18615325 ON n1719 CANCELLED AT 2025-07-26T01:52:27 DUE TO TIME LIMIT ***
⚠ Job 18615320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:50:40,191) - Starting with JobEnvironment(job_id=18615320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:50:40,194) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_submitted.pkl
slurmstepd: error: *** JOB 18615320 ON n1720 CANCELLED AT 2025-07-26T01:50:58 DUE TO TIME LIMIT ***
⚠ Job 18615323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:51:36,163) - Starting with JobEnvironment(job_id=18615323, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:51:36,165) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_submitted.pkl
slurmstepd: error: *** JOB 18615323 ON n1719 CANCELLED AT 2025-07-26T01:51:57 DUE TO TIME LIMIT ***
⚠ Job 18616081 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616081, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_submitted.pkl
slurmstepd: error: *** JOB 18616081 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616078 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616078, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_submitted.pkl
slurmstepd: error: *** JOB 18616078 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,012) - Starting with JobEnvironment(job_id=18616079, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,014) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_submitted.pkl
slurmstepd: error: *** JOB 18616079 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616077 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616077, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_submitted.pkl
slurmstepd: error: *** JOB 18616077 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616071 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616071, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_submitted.pkl
slurmstepd: error: *** JOB 18616071 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616076 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616076, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_submitted.pkl
slurmstepd: error: *** JOB 18616076 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616080 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,012) - Starting with JobEnvironment(job_id=18616080, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_submitted.pkl
slurmstepd: error: *** JOB 18616080 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616069 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616069, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_submitted.pkl
slurmstepd: error: *** JOB 18616069 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616075, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_submitted.pkl
slurmstepd: error: *** JOB 18616075 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616072 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616072, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_submitted.pkl
slurmstepd: error: *** JOB 18616072 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616083 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,010) - Starting with JobEnvironment(job_id=18616083, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,012) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_submitted.pkl
slurmstepd: error: *** JOB 18616083 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616073, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_submitted.pkl
slurmstepd: error: *** JOB 18616073 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616082 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616082, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_submitted.pkl
slurmstepd: error: *** JOB 18616082 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616070 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616070, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_submitted.pkl
slurmstepd: error: *** JOB 18616070 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616074 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616074, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_submitted.pkl
slurmstepd: error: *** JOB 18616074 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616068, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_submitted.pkl
slurmstepd: error: *** JOB 18616068 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616157 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,733) - Starting with JobEnvironment(job_id=18616157, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_submitted.pkl
slurmstepd: error: *** JOB 18616157 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616155 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,227) - Starting with JobEnvironment(job_id=18616155, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,229) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_submitted.pkl
slurmstepd: error: *** JOB 18616155 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616159 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,732) - Starting with JobEnvironment(job_id=18616159, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_submitted.pkl
slurmstepd: error: *** JOB 18616159 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616150 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,206) - Starting with JobEnvironment(job_id=18616150, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_submitted.pkl
slurmstepd: error: *** JOB 18616150 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616148 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:20:47,074) - Starting with JobEnvironment(job_id=18616148, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:20:47,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_submitted.pkl
slurmstepd: error: *** JOB 18616148 ON n1720 CANCELLED AT 2025-07-26T06:21:03 DUE TO TIME LIMIT ***
⚠ Job 18616160 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,731) - Starting with JobEnvironment(job_id=18616160, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,734) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_submitted.pkl
slurmstepd: error: *** JOB 18616160 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616147 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:20:47,074) - Starting with JobEnvironment(job_id=18616147, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:20:47,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_submitted.pkl
slurmstepd: error: *** JOB 18616147 ON n1720 CANCELLED AT 2025-07-26T06:21:03 DUE TO TIME LIMIT ***
⚠ Job 18616146 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:19:46,566) - Starting with JobEnvironment(job_id=18616146, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:19:46,569) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_submitted.pkl
slurmstepd: error: *** JOB 18616146 ON n1720 CANCELLED AT 2025-07-26T06:20:03 DUE TO TIME LIMIT ***
⚠ Job 18616153 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,160) - Starting with JobEnvironment(job_id=18616153, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,174) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_submitted.pkl
slurmstepd: error: *** JOB 18616153 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616149 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:18,030) - Starting with JobEnvironment(job_id=18616149, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:18,032) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_submitted.pkl
slurmstepd: error: *** JOB 18616149 ON n1720 CANCELLED AT 2025-07-26T06:22:33 DUE TO TIME LIMIT ***
⚠ Job 18616158 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,733) - Starting with JobEnvironment(job_id=18616158, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_submitted.pkl
slurmstepd: error: *** JOB 18616158 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616156 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,206) - Starting with JobEnvironment(job_id=18616156, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_submitted.pkl
slurmstepd: error: *** JOB 18616156 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616152 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,202) - Starting with JobEnvironment(job_id=18616152, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,220) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_submitted.pkl
slurmstepd: error: *** JOB 18616152 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616154 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,205) - Starting with JobEnvironment(job_id=18616154, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,207) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_submitted.pkl
slurmstepd: error: *** JOB 18616154 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616151 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,161) - Starting with JobEnvironment(job_id=18616151, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,163) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_submitted.pkl
slurmstepd: error: *** JOB 18616151 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616170 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,338) - Starting with JobEnvironment(job_id=18616170, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,350) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_submitted.pkl
slurmstepd: error: *** JOB 18616170 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616165 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:36:58,253) - Starting with JobEnvironment(job_id=18616165, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:36:58,256) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_submitted.pkl
slurmstepd: error: *** JOB 18616165 ON n1720 CANCELLED AT 2025-07-26T06:37:03 DUE TO TIME LIMIT ***
⚠ Job 18616172 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:59,210) - Starting with JobEnvironment(job_id=18616172, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:59,214) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_submitted.pkl
slurmstepd: error: *** JOB 18616172 ON n1720 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616167 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,294) - Starting with JobEnvironment(job_id=18616167, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,297) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_submitted.pkl
slurmstepd: error: *** JOB 18616167 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616171 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,818) - Starting with JobEnvironment(job_id=18616171, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,837) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_submitted.pkl
slurmstepd: error: *** JOB 18616171 ON n1720 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616162 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:32:25,166) - Starting with JobEnvironment(job_id=18616162, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:32:25,180) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_submitted.pkl
slurmstepd: error: *** JOB 18616162 ON n1720 CANCELLED AT 2025-07-26T06:32:33 DUE TO TIME LIMIT ***
⚠ Job 18616176 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:38:28,647) - Starting with JobEnvironment(job_id=18616176, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:38:28,650) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_submitted.pkl
slurmstepd: error: *** JOB 18616176 ON n1719 CANCELLED AT 2025-07-26T06:38:33 DUE TO TIME LIMIT ***
⚠ Job 18616169 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,297) - Starting with JobEnvironment(job_id=18616169, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,309) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_submitted.pkl
slurmstepd: error: *** JOB 18616169 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616174 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,756) - Starting with JobEnvironment(job_id=18616174, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_submitted.pkl
slurmstepd: error: *** JOB 18616174 ON n1719 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616175 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:38:28,648) - Starting with JobEnvironment(job_id=18616175, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:38:28,651) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_submitted.pkl
slurmstepd: error: *** JOB 18616175 ON n1719 CANCELLED AT 2025-07-26T06:38:33 DUE TO TIME LIMIT ***
⚠ Job 18616161 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:30:54,138) - Starting with JobEnvironment(job_id=18616161, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:30:54,140) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_submitted.pkl
slurmstepd: error: *** JOB 18616161 ON n1720 CANCELLED AT 2025-07-26T06:31:03 DUE TO TIME LIMIT ***
⚠ Job 18616168 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,295) - Starting with JobEnvironment(job_id=18616168, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,319) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_submitted.pkl
slurmstepd: error: *** JOB 18616168 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616173 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,756) - Starting with JobEnvironment(job_id=18616173, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_submitted.pkl
slurmstepd: error: *** JOB 18616173 ON n1719 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616163 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:33:25,757) - Starting with JobEnvironment(job_id=18616163, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:33:25,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_submitted.pkl
slurmstepd: error: *** JOB 18616163 ON n1720 CANCELLED AT 2025-07-26T06:33:33 DUE TO TIME LIMIT ***
⚠ Job 18616177 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:44:02,655) - Starting with JobEnvironment(job_id=18616177, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:44:02,657) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_submitted.pkl
slurmstepd: error: *** JOB 18616177 ON n1719 CANCELLED AT 2025-07-26T06:44:03 DUE TO TIME LIMIT ***
⚠ Job 18616178 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:45:03,317) - Starting with JobEnvironment(job_id=18616178, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:45:03,320) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_submitted.pkl
slurmstepd: error: *** JOB 18616178 ON n1719 CANCELLED AT 2025-07-26T06:45:03 DUE TO TIME LIMIT ***
To cancel, press CTRLc, then run 'scancel 18601847'
⠋Importing logging...⠋Importing warnings...⠋Importing argparse...⠋Importing datetime...⠋Importing dataclass...⠋Importing hashlib...⠋Importing socket...⠋Importing stat...⠋Importing pwd...⠋Importing signal...⠋Importing base64...⠋Importing json...⠋Importing yaml...⠋Importing toml...⠋Importing csv...⠋Importing ast...⠋Importing rich.table...⠋Importing rich print...⠋Importing rich.pretty...⠋Importing rich.prompt...⠋Importing types.FunctionType...⠋Importing typing...⠋Importing ThreadPoolExecutor...⠙Importing submitit.LocalExecutor...⠋Importing submitit.Job...⠋Importing importlib.util...⠋Importing inspect...⠋Importing platform...⠋Importing inspect frame info...⠋Importing pathlib.Path...⠋Importing uuid...⠋Importing traceback...⠋Importing cowsay...⠋Importing psutil...⠋Importing shutil...⠋Importing itertools.combinations...⠋Importing os.listdir...⠋Importing os.path...⠋Importing PIL.Image...⠋Importing sixel...⠋Importing subprocess...⠋Importing tqdm...⠏Importing beartype...⠋Importing statistics...⠋Trying to import pyfiglet...⠹Importing helpers...⠋Parsing arguments...⠹Importing torch...⠋Importing numpy...⠋Importing collections...⠏Importing ax...⠋Importing ax.core.generator_run...⠋Importing Cont_X_trans and Y_trans from ax.modelbridge.registry...⠋Importing ax.core.arm...⠋Importing ax.core.objective...⠋Importing ax.core.Metric...⠋Importing ax.exceptions.core...⠋Importing ax.exceptions.generation_strategy...⠋Importing CORE_DECODER_REGISTRY...⠋Trying ax.generation_strategy.generation_node...⠋Importing GenerationStep, GenerationStrategy from generation_strategy...⠋Importing GenerationNode from generation_node...⠋Importing ExternalGenerationNode...⠋Importing MaxTrials...⠋Importing GeneratorSpec...⠋Importing Models from ax.modelbridge.registry...⠋Importing get_pending_observation_features...⠋Importing load_experiment...⠋Importing save_experiment...⠋Importing save_experiment_to_db...⠋Importing TrialStatus...⠋Importing Data...⠋Importing Experiment...⠋Importing parameter types...⠋Importing TParameterization...⠋Importing pandas...⠋Importing AxClient and ObjectiveProperties...⠋Importing RandomForestRegressor...⠋Importing botorch...⠋Importing submitit...⠋Importing ax logger...⠋Importing SQL-Storage-Stuff...
Run-UUID: 76381440-17db-4436-855f-fb3a0e1389bd
___________________________________
| OmniOpt2 - When in doubt, optimize! |
===================================
\
\
\
\
.-=-==--==--.
..-==" ,'o`) `.
,' `"' \
: ( `.__...._
| ) / `-=-.
: ,vv.-._ / / `---==-._
\/\/\/VV ^ d88`;' / `.
`` ^/d88P!' / , `._
^/ !' ,. , / "-,,__,,--'""""-.
^/ !' ,' \ . .( ( _ ) ) ) ) ))_,-.\
^(__ ,!',"' ;:+.:%:a. \:.. . ,' ) ) ) ) ,"' '
',,,'',' /o:::":%:%a. \:.:.: . ) ) _,'
"""' ;':::'' `+%%%a._ \%:%| ;.). _,-""
,-='_.-' ``:%::) )%:| /:._,"
(/(/" ," ,'_,'%%%: (_,'
( (//(`.___; \
\ \ ` `
`. `. `. :
\. . .\ : . . . :
\. . .: `.. . .:
`..:.:\ \:...\
;:.:.; ::...:
):%:: :::::;
__,::%:( :::::
,;:%%%%%%%: ;:%::
;,--""-.`\ ,=--':%:%:\
/" "| /-".:%%%%%%%\
;,-"'`)%%)
/" "|
⠋Writing worker creation log...
omniopt --partition=barnard --experiment_name=mnist_cpu --mem_gb=5 --time=2880 --worker_timeout=120 --max_eval=500 --num_parallel_jobs=50 --gpus=0 --num_random_steps=50 --follow --live_share --send_anonymized_usage_stats --result_names VAL_ACC=max --run_program=cHl0aG9uMyAudGVzdHMvbW5pc3QvdHJhaW4gLS1lcG9jaHMgJWVwb2NocyAtLWxlYXJuaW5nX3JhdGUgJWxyIC0tYmF0Y2hfc2l6ZSAlYmF0Y2hfc2l6ZSAtLWhpZGRlbl9zaXplICVoaWRkZW5fc2l6ZSAtLWRyb3BvdXQgJWRyb3BvdXQgLS1hY3RpdmF0aW9uICVhY3RpdmF0aW9uIC0tbnVtX2RlbnNlX2xheWVycyAlbnVtX2RlbnNlX2xheWVycyAtLWluaXQgJWluaXQgLS13ZWlnaHRfZGVjYXkgJXdlaWdodF9kZWNheQ== --cpus_per_task=1 --nodes_per_job=1 --revert_to_random_when_seemingly_exhausted --model=BOTORCH_MODULAR --n_estimators_randomforest=100 --run_mode=local --occ_type=euclid --main_process_gb=8 --max_nr_of_zero_results=50 --slurm_signal_delay_s=0 --max_failed_jobs=0 --max_attempts_for_generation=20 --num_restarts=20 --raw_samples=1024 --max_abandoned_retrial=20 --max_num_of_parallel_sruns=16 --generate_all_jobs_at_once --parameter epochs range 10 200 int false --parameter lr range 0.0001 0.1 float false --parameter batch_size range 8 4096 int false --parameter hidden_size range 8 8192 int false --parameter dropout range 0 0.5 float false --parameter activation fixed leaky_relu --parameter num_dense_layers range 1 4 int false --parameter init fixed normal --parameter weight_decay range 0 1 float false --ui_url aHR0cHM6Ly9pbWFnZXNlZy5zY2Fkcy5kZS9vbW5pYXgvZ3VpP3BhcnRpdGlvbj1iYXJuYXJkJmV4cGVyaW1lbnRfbmFtZT1tbmlzdF9jcHUmcmVzZXJ2YXRpb249JmFjY291bnQ9Jm1lbV9nYj01JnRpbWU9Mjg4MCZ3b3JrZXJfdGltZW91dD0xMjAmbWF4X2V2YWw9NTAwJm51bV9wYXJhbGxlbF9qb2JzPTUwJmdwdXM9MCZudW1fcmFuZG9tX3N0ZXBzPTUwJmZvbGxvdz0xJmxpdmVfc2hhcmU9MSZzZW5kX2Fub255bWl6ZWRfdXNhZ2Vfc3RhdHM9MSZjb25zdHJhaW50cz0mcmVzdWx0X25hbWVzPVZBTF9BQ0MlM0RtYXgmcnVuX3Byb2dyYW09cHl0aG9uMyUyMC50ZXN0cyUyRm1uaXN0JTJGdHJhaW4lMjAtLWVwb2NocyUyMCUyNWVwb2NocyUyMC0tbGVhcm5pbmdfcmF0ZSUyMCUyNWxyJTIwLS1iYXRjaF9zaXplJTIwJTI1YmF0Y2hfc2l6ZSUyMC0taGlkZGVuX3NpemUlMjAlMjVoaWRkZW5fc2l6ZSUyMC0tZHJvcG91dCUyMCUyNWRyb3BvdXQlMjAtLWFjdGl2YXRpb24lMjAlMjVhY3RpdmF0aW9uJTIwLS1udW1fZGVuc2VfbGF5ZXJzJTIwJTI1bnVtX2RlbnNlX2xheWVycyUyMC0taW5pdCUyMCUyNWluaXQlMjAtLXdlaWdodF9kZWNheSUyMCUyNXdlaWdodF9kZWNheSZjcHVzX3Blcl90YXNrPTEmbm9kZXNfcGVyX2pvYj0xJnNlZWQ9JmRyeXJ1bj0wJmRlYnVnPTAmcmV2ZXJ0X3RvX3JhbmRvbV93aGVuX3NlZW1pbmdseV9leGhhdXN0ZWQ9MSZncmlkc2VhcmNoPTAmbW9kZWw9Qk9UT1JDSF9NT0RVTEFSJmV4dGVybmFsX2dlbmVyYXRvcj0mbl9lc3RpbWF0b3JzX3JhbmRvbWZvcmVzdD0xMDAmaW5zdGFsbGF0aW9uX21ldGhvZD1jbG9uZSZydW5fbW9kZT1sb2NhbCZkaXNhYmxlX3RxZG09MCZ2ZXJib3NlX3RxZG09MCZmb3JjZV9sb2NhbF9leGVjdXRpb249MCZhdXRvX2V4Y2x1ZGVfZGVmZWN0aXZlX2hvc3RzPTAmc2hvd19zaXhlbF9nZW5lcmFsPTAmc2hvd19zaXhlbF90cmlhbF9pbmRleF9yZXN1bHQ9MCZzaG93X3NpeGVsX3NjYXR0ZXI9MCZzaG93X3dvcmtlcl9wZXJjZW50YWdlX3RhYmxlX2F0X2VuZD0wJm9jYz0wJm9jY190eXBlPWV1Y2xpZCZub19zbGVlcD0wJnNsdXJtX3VzZV9zcnVuPTAmdmVyYm9zZV9icmVha19ydW5fc2VhcmNoX3RhYmxlPTAmYWJicmV2aWF0ZV9qb2JfbmFtZXM9MCZtYWluX3Byb2Nlc3NfZ2I9OCZtYXhfbnJfb2ZfemVyb19yZXN1bHRzPTUwJnNsdXJtX3NpZ25hbF9kZWxheV9zPTAmbWF4X2ZhaWxlZF9qb2JzPTAmZXhjbHVkZT0mdXNlcm5hbWU9JmdlbmVyYXRpb25fc3RyYXRlZ3k9JnJvb3RfdmVudl9kaXI9JndvcmtkaXI9JmRvbnRfaml0X2NvbXBpbGU9MCZmaXRfb3V0X29mX2Rlc2lnbj0wJnJlZml0X29uX2N2PTAmc2hvd19nZW5lcmF0ZV90aW1lX3RhYmxlPTAmZG9udF93YXJtX3N0YXJ0X3JlZml0dGluZz0wJm1heF9hdHRlbXB0c19mb3JfZ2VuZXJhdGlvbj0yMCZudW1fcmVzdGFydHM9MjAmcmF3X3NhbXBsZXM9MTAyNCZtYXhfYWJhbmRvbmVkX3JldHJpYWw9MjAmbWF4X251bV9vZl9wYXJhbGxlbF9zcnVucz0xNiZmb3JjZV9jaG9pY2VfZm9yX3Jhbmdlcz0wJm5vX3RyYW5zZm9ybV9pbnB1dHM9MCZmaXRfYWJhbmRvbmVkPTAmbm9fbm9ybWFsaXplX3k9MCZ2ZXJib3NlPTAmZ2VuZXJhdGVfYWxsX2pvYnNfYXRfb25jZT0xJmZsYW1lX2dyYXBoPTAmY2hlY2tvdXRfdG9fbGF0ZXN0X3Rlc3RlZF92ZXJzaW9uPTAmcGFyYW1ldGVyXzBfbmFtZT1lcG9jaHMmcGFyYW1ldGVyXzBfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfMF9taW49MTAmcGFyYW1ldGVyXzBfbWF4PTIwMCZwYXJhbWV0ZXJfMF9udW1iZXJfdHlwZT1pbnQmcGFyYW1ldGVyXzBfbG9nX3NjYWxlPWZhbHNlJnBhcmFtZXRlcl8xX25hbWU9bHImcGFyYW1ldGVyXzFfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfMV9taW49MC4wMDAxJnBhcmFtZXRlcl8xX21heD0wLjEmcGFyYW1ldGVyXzFfbnVtYmVyX3R5cGU9ZmxvYXQmcGFyYW1ldGVyXzFfbG9nX3NjYWxlPWZhbHNlJnBhcmFtZXRlcl8yX25hbWU9YmF0Y2hfc2l6ZSZwYXJhbWV0ZXJfMl90eXBlPXJhbmdlJnBhcmFtZXRlcl8yX21pbj04JnBhcmFtZXRlcl8yX21heD00MDk2JnBhcmFtZXRlcl8yX251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfMl9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzNfbmFtZT1oaWRkZW5fc2l6ZSZwYXJhbWV0ZXJfM190eXBlPXJhbmdlJnBhcmFtZXRlcl8zX21pbj04JnBhcmFtZXRlcl8zX21heD04MTkyJnBhcmFtZXRlcl8zX251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfM19sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzRfbmFtZT1kcm9wb3V0JnBhcmFtZXRlcl80X3R5cGU9cmFuZ2UmcGFyYW1ldGVyXzRfbWluPTAmcGFyYW1ldGVyXzRfbWF4PTAuNSZwYXJhbWV0ZXJfNF9udW1iZXJfdHlwZT1mbG9hdCZwYXJhbWV0ZXJfNF9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzVfbmFtZT1hY3RpdmF0aW9uJnBhcmFtZXRlcl81X3R5cGU9Zml4ZWQmcGFyYW1ldGVyXzVfdmFsdWU9bGVha3lfcmVsdSZwYXJhbWV0ZXJfNl9uYW1lPW51bV9kZW5zZV9sYXllcnMmcGFyYW1ldGVyXzZfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfNl9taW49MSZwYXJhbWV0ZXJfNl9tYXg9NCZwYXJhbWV0ZXJfNl9udW1iZXJfdHlwZT1pbnQmcGFyYW1ldGVyXzZfbG9nX3NjYWxlPWZhbHNlJnBhcmFtZXRlcl83X25hbWU9aW5pdCZwYXJhbWV0ZXJfN190eXBlPWZpeGVkJnBhcmFtZXRlcl83X3ZhbHVlPW5vcm1hbCZwYXJhbWV0ZXJfOF9uYW1lPXdlaWdodF9kZWNheSZwYXJhbWV0ZXJfOF90eXBlPXJhbmdlJnBhcmFtZXRlcl84X21pbj0wJnBhcmFtZXRlcl84X21heD0xJnBhcmFtZXRlcl84X251bWJlcl90eXBlPWZsb2F0JnBhcmFtZXRlcl84X2xvZ19zY2FsZT1mYWxzZSZwYXJ0aXRpb249YmFybmFyZCZudW1fcGFyYW1ldGVycz05
⠋Disabling logging...⠋Setting run folder...⠋Creating folder /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3...⠋Writing revert_to_random_when_seemingly_exhausted file ...⠋Writing username state file...⠋Writing result names file...⠋Writing result min/max file...⠋Saving state files...Run-folder: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3⠋Printing run info...⠋Initializing NVIDIA-Logs...⠋Writing ui_url file if it is present...⠋Writing live_share file if it is present...⠋Writing job_start_time file...⠸Writing git info file...⠋Checking max_eval...⠋Calculating number of steps...⠋Adding excluded nodes...⠋Handling random steps...⠋Initializing ax_client...
[WARNING 07-24 13:38:19] ax.service.ax_client: Selecting a GenerationStrategy when using BatchTrials is in beta. Double check the recommended strategy matches your expectations.
⠋Setting orchestrator...You have 2 CPUs available for the main process.No CUDA devices found.Generation strategy: SOBOL for 50 steps and then BOTORCH_MODULAR for 450 steps.
Run-Program: python3 .tests/mnist/train --epochs %epochs --learning_rate %lr --batch_size %batch_size --hidden_size %hidden_size --dropout %dropout --activation %activation --num_dense_layers %num_dense_layers --init %init --weight_decay %weight_decay
Experiment parameters
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃Name ┃Type ┃Lower bound┃Upper bound┃Values ┃Type ┃Log Scale?┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│epochs │range│10 │200 ││int │No │
│lr │range│0.0001 │0.1 ││float│No │
│batch_size │range│8 │4096 ││int │No │
│hidden_size │range│8 │8192 ││int │No │
│dropout │range│0 │0.5 ││float│No │
│activation │fixed│││leaky_relu│││
│num_dense_layers│range│1 │4 ││int │No │
│init │fixed│││normal │││
│weight_decay │range│0 │1 ││float│No │
└──────────────────┴───────┴─────────────┴─────────────┴────────────┴───────┴────────────┘
Result-Names
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃Result-Name┃Min or max?┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│VAL_ACC │ max│
└─────────────┴─────────────┘
See https://imageseg.scads.de/omniax/share?user_id=pwinkler&experiment_name=mnist_cpu&run_nr=3 for live-results.
█▀▀▀▀▀█ ███ ▀ ▀▀█ █▄▄█▄▀ ▀▀ █▀▀▀▀▀█
█ ███ █ █▄▀█ ▄▄ ▀██▀▀▀▀ ▄█▀▀ █ ███ █
█ ▀▀▀ █ ▄ ▄▄▀▄ ▀▀▀▄ █ ▀ ▀▄▀█ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀▄▀ █ ▀ ▀▄█▄█▄▀ █ █▄█ ▀▀▀▀▀▀▀
▀▀▄▄▀█▀ ▄▀▄ ▀█▄▀▄▄ █▀▄ ▄▄▄ ▀▄▀██▀
█▄ █ ▀ ▄██▀ ██ ██ ▄█▄█ █▄▀▄▀▀█▄▀▄
██▀▄▄▀▀▀▀▄ ▀▄▀▄██▀▀ ▀▀▄ ▄▄█▄▀▀ ▄▄█▄▀
▀▄ ▀ ▀▀ █ ▄▀▄▀███▄▄█ ██ ▄▀▀▀ ▄█▄▄▄
▄▄▀ ▄▄▀▄█ ▄▀▄▀█▄██▄ ▀▀██▄▄ █▄▀▀█▄▄█▄▀
▄▀▀▀█▀▀▀ █▀█▀█▀▀▀██ █▄██▄▀ █ ▄ █▀ ▄▄
▀▀▀▄█▀▀▄█▄ ▀█▄▄ █▄▀▀▀ ▄▀▄▄█▄█▀▀▄ ██▀
▄ ▄▀ ▀▀ █▀▀ ▄ ▄█ ▄▀ █▄▄█▀ █▀ ▀▄█ █▄
██▀▀▀ ▀▄▄▄█▀▄▀▀▄██▄ ▀▀▀█▀ ▀█▄ ▀▀▄▄██▀
▄▄▀▄▀▄▀▄██▀█▀███▄ ▀█ ▀█▀█▀ ▄█▀▀ █▄
▀▀▀▀▀ ▀ ██▄ ▀█▄▄█▀ ▀██▀ ▄ ▀█▀▀▀█ ▀
█▀▀▀▀▀█ ▄▄ ▄ ▄▀▄▄▄▄ ██▀ █▄█ ▀ █ █▄▄
█ ███ █ ▀██▀▄▀▀▄▄▀ ▀ ▀█▄ ▄ ▄▀██▀▀▄▀▄▄
█ ▀▀▀ █ ▄ █▀█▀▀ █▀ █████▀▄▄▀▀▀█ ▄▄
▀▀▀▀▀▀▀ ▀▀ ▀▀ ▀▀ ▀▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀
Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs : 3%|░░░░░░░░░░| 17/500 [1:06:05<38:44:56, 288.81s/it]
Runtime (end): 1 hours 11 minutes and 13 seconds, PID: 1105498
Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs : 4%|░░░░░░░░░░| 20/500 [2:03:31<101:24:52, 760.61s/it]
⚠ Job 18601920 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601920, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601920/18601920_submitted.pkl
slurmstepd: error: *** JOB 18601920 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601916 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,597) - Starting with JobEnvironment(job_id=18601916, hostname=n1598, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601916/18601916_submitted.pkl
slurmstepd: error: *** JOB 18601916 ON n1598 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601921 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601921, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601921/18601921_submitted.pkl
slurmstepd: error: *** JOB 18601921 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601919 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601919, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601919/18601919_submitted.pkl
slurmstepd: error: *** JOB 18601919 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601939 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,748) - Starting with JobEnvironment(job_id=18601939, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,750) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601939/18601939_submitted.pkl
slurmstepd: error: *** JOB 18601939 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601914 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,079) - Starting with JobEnvironment(job_id=18601914, hostname=n1626, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,081) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_submitted.pkl
slurmstepd: error: *** JOB 18601914 ON n1626 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601964 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,763) - Starting with JobEnvironment(job_id=18601964, hostname=n1585, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,765) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601964/18601964_submitted.pkl
slurmstepd: error: *** JOB 18601964 ON n1585 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601923 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601923, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601923/18601923_submitted.pkl
slurmstepd: error: *** JOB 18601923 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601962 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601962, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601962/18601962_submitted.pkl
slurmstepd: error: *** JOB 18601962 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601940 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601940, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601940/18601940_submitted.pkl
slurmstepd: error: *** JOB 18601940 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601952 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601952, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,710) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601952/18601952_submitted.pkl
slurmstepd: error: *** JOB 18601952 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601956 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601956, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,709) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601956/18601956_submitted.pkl
slurmstepd: error: *** JOB 18601956 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601935 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:10,179) - Starting with JobEnvironment(job_id=18601935, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:10,181) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601935/18601935_submitted.pkl
slurmstepd: error: *** JOB 18601935 ON n1588 CANCELLED AT 2025-07-24T15:51:35 DUE TO TIME LIMIT ***
⚠ Job 18601948 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:15,307) - Starting with JobEnvironment(job_id=18601948, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:15,310) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601948/18601948_submitted.pkl
slurmstepd: error: *** JOB 18601948 ON n1587 CANCELLED AT 2025-07-24T15:52:35 DUE TO TIME LIMIT ***
⚠ Job 18601922 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:05,693) - Starting with JobEnvironment(job_id=18601922, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:05,695) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601922/18601922_submitted.pkl
slurmstepd: error: *** JOB 18601922 ON n1588 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601958 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601958, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601958/18601958_submitted.pkl
slurmstepd: error: *** JOB 18601958 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601975 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:54:25,590) - Starting with JobEnvironment(job_id=18601975, hostname=n1577, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:54:25,592) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601975/18601975_submitted.pkl
slurmstepd: error: *** JOB 18601975 ON n1577 CANCELLED AT 2025-07-24T15:54:35 DUE TO TIME LIMIT ***
⚠ Job 18601917 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,705) - Starting with JobEnvironment(job_id=18601917, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,708) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601917/18601917_submitted.pkl
slurmstepd: error: *** JOB 18601917 ON n1589 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601937 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601937, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601937/18601937_submitted.pkl
slurmstepd: error: *** JOB 18601937 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601963 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,678) - Starting with JobEnvironment(job_id=18601963, hostname=n1586, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,680) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601963/18601963_submitted.pkl
slurmstepd: error: *** JOB 18601963 ON n1586 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601938 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,747) - Starting with JobEnvironment(job_id=18601938, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,749) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601938/18601938_submitted.pkl
slurmstepd: error: *** JOB 18601938 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601966 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:53,954) - Starting with JobEnvironment(job_id=18601966, hostname=n1577, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:53,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601966/18601966_submitted.pkl
slurmstepd: error: *** JOB 18601966 ON n1577 CANCELLED AT 2025-07-24T15:54:05 DUE TO TIME LIMIT ***
⚠ Job 18601932 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:10,179) - Starting with JobEnvironment(job_id=18601932, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:10,181) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601932/18601932_submitted.pkl
slurmstepd: error: *** JOB 18601932 ON n1588 CANCELLED AT 2025-07-24T15:51:35 DUE TO TIME LIMIT ***
⚠ Job 18601953 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:52:47,708) - Starting with JobEnvironment(job_id=18601953, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:52:47,710) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601953/18601953_submitted.pkl
slurmstepd: error: *** JOB 18601953 ON n1587 CANCELLED AT 2025-07-24T15:53:05 DUE TO TIME LIMIT ***
⚠ Job 18601929 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:37,733) - Starting with JobEnvironment(job_id=18601929, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:37,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601929/18601929_submitted.pkl
slurmstepd: error: *** JOB 18601929 ON n1588 CANCELLED AT 2025-07-24T15:51:05 DUE TO TIME LIMIT ***
⚠ Job 18601960 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601960, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601960/18601960_submitted.pkl
slurmstepd: error: *** JOB 18601960 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
⚠ Job 18601927 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:37,733) - Starting with JobEnvironment(job_id=18601927, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:37,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601927/18601927_submitted.pkl
slurmstepd: error: *** JOB 18601927 ON n1588 CANCELLED AT 2025-07-24T15:51:05 DUE TO TIME LIMIT ***
⚠ Job 18601945 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:51:42,749) - Starting with JobEnvironment(job_id=18601945, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:51:42,751) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601945/18601945_submitted.pkl
slurmstepd: error: *** JOB 18601945 ON n1588 CANCELLED AT 2025-07-24T15:52:06 DUE TO TIME LIMIT ***
⚠ Job 18601915 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:50:06,823) - Starting with JobEnvironment(job_id=18601915, hostname=n1618, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,825) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601915/18601915_submitted.pkl
slurmstepd: error: *** JOB 18601915 ON n1618 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***
⚠ Job 18601959 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_0_log.out
----------------------------------------
submitit INFO (2025-07-24 13:53:20,274) - Starting with JobEnvironment(job_id=18601959, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:53:20,276) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601959/18601959_submitted.pkl
slurmstepd: error: *** JOB 18601959 ON n1587 CANCELLED AT 2025-07-24T15:53:35 DUE TO TIME LIMIT ***
Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs : 8%|▒░░░░░░░░░| 38/500 [3:59:25<39:46:37, 309.95s/it]
⚠ Job 18602999 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,631) - Starting with JobEnvironment(job_id=18602999, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,634) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602999/18602999_submitted.pkl
slurmstepd: error: *** JOB 18602999 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603030 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603030, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603030/18603030_submitted.pkl
slurmstepd: error: *** JOB 18603030 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602991 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602991, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602991/18602991_submitted.pkl
slurmstepd: error: *** JOB 18602991 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603066, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,597) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603066/18603066_submitted.pkl
slurmstepd: error: *** JOB 18603066 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603031 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603031, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603031/18603031_submitted.pkl
slurmstepd: error: *** JOB 18603031 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602985 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,898) - Starting with JobEnvironment(job_id=18602985, hostname=n1609, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,900) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602985/18602985_submitted.pkl
slurmstepd: error: *** JOB 18602985 ON n1609 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603029 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603029, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603029/18603029_submitted.pkl
slurmstepd: error: *** JOB 18603029 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603051 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:08,011) - Starting with JobEnvironment(job_id=18603051, hostname=n1408, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:08,034) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603051/18603051_submitted.pkl
slurmstepd: error: *** JOB 18603051 ON n1408 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603112 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,368) - Starting with JobEnvironment(job_id=18603112, hostname=n1432, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,370) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603112/18603112_submitted.pkl
slurmstepd: error: *** JOB 18603112 ON n1432 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18602998 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,714) - Starting with JobEnvironment(job_id=18602998, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,717) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602998/18602998_submitted.pkl
slurmstepd: error: *** JOB 18602998 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603131 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603131, hostname=n1397, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,401) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603131/18603131_submitted.pkl
slurmstepd: error: *** JOB 18603131 ON n1397 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18602987 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602987, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,714) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602987/18602987_submitted.pkl
slurmstepd: error: *** JOB 18602987 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
⚠ Job 18603032 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:33,328) - Starting with JobEnvironment(job_id=18603032, hostname=n1439, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:33,330) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603032/18603032_submitted.pkl
slurmstepd: error: *** JOB 18603032 ON n1439 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603130 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603130, hostname=n1397, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,400) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603130/18603130_submitted.pkl
slurmstepd: error: *** JOB 18603130 ON n1397 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603027 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603027, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603027/18603027_submitted.pkl
slurmstepd: error: *** JOB 18603027 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603095 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603095, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603095/18603095_submitted.pkl
slurmstepd: error: *** JOB 18603095 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18602993 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602993, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602993/18602993_submitted.pkl
slurmstepd: error: *** JOB 18602993 ON n1606 CANCELLED AT 2025-07-24T18:27:56 DUE TO TIME LIMIT ***
⚠ Job 18603108 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603108, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,597) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603108/18603108_submitted.pkl
slurmstepd: error: *** JOB 18603108 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603024 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:33,558) - Starting with JobEnvironment(job_id=18603024, hostname=n1539, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:33,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603024/18603024_submitted.pkl
slurmstepd: error: *** JOB 18603024 ON n1539 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18603115 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,652) - Starting with JobEnvironment(job_id=18603115, hostname=n1398, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,654) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603115/18603115_submitted.pkl
slurmstepd: error: *** JOB 18603115 ON n1398 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603107 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603107, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603107/18603107_submitted.pkl
slurmstepd: error: *** JOB 18603107 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603113 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:40,398) - Starting with JobEnvironment(job_id=18603113, hostname=n1428, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:40,401) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603113/18603113_submitted.pkl
slurmstepd: error: *** JOB 18603113 ON n1428 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603019 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:00,514) - Starting with JobEnvironment(job_id=18603019, hostname=n1540, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:00,516) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603019/18603019_submitted.pkl
slurmstepd: error: *** JOB 18603019 ON n1540 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603109 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:07,595) - Starting with JobEnvironment(job_id=18603109, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:07,596) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603109/18603109_submitted.pkl
slurmstepd: error: *** JOB 18603109 ON n1399 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18603049 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:08,011) - Starting with JobEnvironment(job_id=18603049, hostname=n1408, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:08,026) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603049/18603049_submitted.pkl
slurmstepd: error: *** JOB 18603049 ON n1408 CANCELLED AT 2025-07-24T18:29:24 DUE TO TIME LIMIT ***
⚠ Job 18602995 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,976) - Starting with JobEnvironment(job_id=18602995, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,980) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602995/18602995_submitted.pkl
slurmstepd: error: *** JOB 18602995 ON n1589 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18602996 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,710) - Starting with JobEnvironment(job_id=18602996, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,712) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602996/18602996_submitted.pkl
slurmstepd: error: *** JOB 18602996 ON n1588 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18603114 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:29:39,662) - Starting with JobEnvironment(job_id=18603114, hostname=n1399, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:29:39,665) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603114/18603114_submitted.pkl
slurmstepd: error: *** JOB 18603114 ON n1399 CANCELLED AT 2025-07-24T18:29:54 DUE TO TIME LIMIT ***
⚠ Job 18603028 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:28:32,801) - Starting with JobEnvironment(job_id=18603028, hostname=n1443, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:28:32,803) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603028/18603028_submitted.pkl
slurmstepd: error: *** JOB 18603028 ON n1443 CANCELLED AT 2025-07-24T18:28:55 DUE TO TIME LIMIT ***
⚠ Job 18602992 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,712) - Starting with JobEnvironment(job_id=18602992, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,714) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602992/18602992_submitted.pkl
slurmstepd: error: *** JOB 18602992 ON n1606 CANCELLED AT 2025-07-24T18:27:56 DUE TO TIME LIMIT ***
⚠ Job 18603018 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:59,730) - Starting with JobEnvironment(job_id=18603018, hostname=n1587, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:59,733) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603018/18603018_submitted.pkl
slurmstepd: error: *** JOB 18603018 ON n1587 CANCELLED AT 2025-07-24T18:28:25 DUE TO TIME LIMIT ***
⚠ Job 18602990 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_0_log.out
----------------------------------------
submitit INFO (2025-07-24 16:27:28,711) - Starting with JobEnvironment(job_id=18602990, hostname=n1606, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 16:27:28,713) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18602990/18602990_submitted.pkl
slurmstepd: error: *** JOB 18602990 ON n1606 CANCELLED AT 2025-07-24T18:27:55 DUE TO TIME LIMIT ***
Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs : 11%|█░░░░░░░░░| 57/500 [7:22:04<52:07:11, 423.55s/it]
⚠ Job 18603858 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603858, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603858/18603858_submitted.pkl
slurmstepd: error: *** JOB 18603858 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603855 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:08,463) - Starting with JobEnvironment(job_id=18603855, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:08,466) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603855/18603855_submitted.pkl
slurmstepd: error: *** JOB 18603855 ON n1720 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603869 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:04:24,717) - Starting with JobEnvironment(job_id=18603869, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:04:24,719) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603869/18603869_submitted.pkl
slurmstepd: error: *** JOB 18603869 ON n1707 CANCELLED AT 2025-07-24T21:04:42 DUE TO TIME LIMIT ***
⚠ Job 18603857 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603857, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603857/18603857_submitted.pkl
slurmstepd: error: *** JOB 18603857 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603859 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603859, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603859/18603859_submitted.pkl
slurmstepd: error: *** JOB 18603859 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603849 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:01:37,133) - Starting with JobEnvironment(job_id=18603849, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:01:37,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603849/18603849_submitted.pkl
slurmstepd: error: *** JOB 18603849 ON n1720 CANCELLED AT 2025-07-24T21:01:42 DUE TO TIME LIMIT ***
⚠ Job 18603856 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:08,463) - Starting with JobEnvironment(job_id=18603856, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:08,465) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603856/18603856_submitted.pkl
slurmstepd: error: *** JOB 18603856 ON n1720 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603851 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:01:37,133) - Starting with JobEnvironment(job_id=18603851, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:01:37,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603851/18603851_submitted.pkl
slurmstepd: error: *** JOB 18603851 ON n1720 CANCELLED AT 2025-07-24T21:01:42 DUE TO TIME LIMIT ***
⚠ Job 18603862 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603862, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603862/18603862_submitted.pkl
slurmstepd: error: *** JOB 18603862 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
⚠ Job 18603864 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:42,561) - Starting with JobEnvironment(job_id=18603864, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:42,564) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603864/18603864_submitted.pkl
slurmstepd: error: *** JOB 18603864 ON n1719 CANCELLED AT 2025-07-24T21:02:42 DUE TO TIME LIMIT ***
⚠ Job 18603867 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:03:54,459) - Starting with JobEnvironment(job_id=18603867, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:03:54,461) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603867/18603867_submitted.pkl
slurmstepd: error: *** JOB 18603867 ON n1707 CANCELLED AT 2025-07-24T21:04:12 DUE TO TIME LIMIT ***
⚠ Job 18603861 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:02:09,327) - Starting with JobEnvironment(job_id=18603861, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:02:09,329) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603861/18603861_submitted.pkl
slurmstepd: error: *** JOB 18603861 ON n1719 CANCELLED AT 2025-07-24T21:02:13 DUE TO TIME LIMIT ***
Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs : 11%|█░░░░░░░░░| 57/500 [7:29:53<52:07:11, 423.55s/it]
⚠ Job 18603878 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603878, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603878/18603878_submitted.pkl
slurmstepd: error: *** JOB 18603878 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603887 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:42,364) - Starting with JobEnvironment(job_id=18603887, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:42,366) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603887/18603887_submitted.pkl
slurmstepd: error: *** JOB 18603887 ON n1695 CANCELLED AT 2025-07-24T21:07:42 DUE TO TIME LIMIT ***
⚠ Job 18603921 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:34,494) - Starting with JobEnvironment(job_id=18603921, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:34,496) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603921/18603921_submitted.pkl
slurmstepd: error: *** JOB 18603921 ON n1693 CANCELLED AT 2025-07-24T21:11:42 DUE TO TIME LIMIT ***
⚠ Job 18603881 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603881, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603881/18603881_submitted.pkl
slurmstepd: error: *** JOB 18603881 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603915 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:01,516) - Starting with JobEnvironment(job_id=18603915, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:01,518) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603915/18603915_submitted.pkl
slurmstepd: error: *** JOB 18603915 ON n1693 CANCELLED AT 2025-07-24T21:11:12 DUE TO TIME LIMIT ***
⚠ Job 18603899 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:21,987) - Starting with JobEnvironment(job_id=18603899, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:21,989) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603899/18603899_submitted.pkl
slurmstepd: error: *** JOB 18603899 ON n1695 CANCELLED AT 2025-07-24T21:09:42 DUE TO TIME LIMIT ***
⚠ Job 18603903 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603903, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603903/18603903_submitted.pkl
slurmstepd: error: *** JOB 18603903 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603906 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603906, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603906/18603906_submitted.pkl
slurmstepd: error: *** JOB 18603906 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603876 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:03,133) - Starting with JobEnvironment(job_id=18603876, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:03,135) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603876/18603876_submitted.pkl
slurmstepd: error: *** JOB 18603876 ON n1707 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603892 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:08:15,382) - Starting with JobEnvironment(job_id=18603892, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:08:15,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603892/18603892_submitted.pkl
slurmstepd: error: *** JOB 18603892 ON n1695 CANCELLED AT 2025-07-24T21:08:43 DUE TO TIME LIMIT ***
⚠ Job 18603908 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:00,476) - Starting with JobEnvironment(job_id=18603908, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:00,478) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603908/18603908_submitted.pkl
slurmstepd: error: *** JOB 18603908 ON n1694 CANCELLED AT 2025-07-24T21:11:12 DUE TO TIME LIMIT ***
⚠ Job 18603893 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:08:15,382) - Starting with JobEnvironment(job_id=18603893, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:08:15,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603893/18603893_submitted.pkl
slurmstepd: error: *** JOB 18603893 ON n1695 CANCELLED AT 2025-07-24T21:08:43 DUE TO TIME LIMIT ***
⚠ Job 18603902 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603902, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603902/18603902_submitted.pkl
slurmstepd: error: *** JOB 18603902 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603886 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:09,087) - Starting with JobEnvironment(job_id=18603886, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:09,089) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603886/18603886_submitted.pkl
slurmstepd: error: *** JOB 18603886 ON n1696 CANCELLED AT 2025-07-24T21:07:13 DUE TO TIME LIMIT ***
⚠ Job 18603920 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:11:34,494) - Starting with JobEnvironment(job_id=18603920, hostname=n1693, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:11:34,496) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603920/18603920_submitted.pkl
slurmstepd: error: *** JOB 18603920 ON n1693 CANCELLED AT 2025-07-24T21:11:42 DUE TO TIME LIMIT ***
⚠ Job 18603904 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:55,252) - Starting with JobEnvironment(job_id=18603904, hostname=n1694, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:55,254) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603904/18603904_submitted.pkl
slurmstepd: error: *** JOB 18603904 ON n1694 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603877 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:06:04,382) - Starting with JobEnvironment(job_id=18603877, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:06:04,384) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603877/18603877_submitted.pkl
slurmstepd: error: *** JOB 18603877 ON n1696 CANCELLED AT 2025-07-24T21:06:13 DUE TO TIME LIMIT ***
⚠ Job 18603901 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:09:54,359) - Starting with JobEnvironment(job_id=18603901, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:09:54,361) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603901/18603901_submitted.pkl
slurmstepd: error: *** JOB 18603901 ON n1695 CANCELLED AT 2025-07-24T21:10:12 DUE TO TIME LIMIT ***
⚠ Job 18603889 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_0_log.out
----------------------------------------
submitit INFO (2025-07-24 19:07:42,364) - Starting with JobEnvironment(job_id=18603889, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 19:07:42,366) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18603889/18603889_submitted.pkl
slurmstepd: error: *** JOB 18603889 ON n1695 CANCELLED AT 2025-07-24T21:07:42 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, waiting for 4 jobs, finished 4 jobs : 21%|██░░░░░░░░| 107/500 [8:58:12<8:05:11, 74.08s/it][WARNING 07-24 22:43:17] ax.modelbridge.base: TorchAdapter(model=BoTorchGenerator) was not able to generate 50 unique candidates. Generated arms have the following weights, as there are repeats:
[2.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs : 30%|███░░░░░░░| 151/500 [11:29:21<13:44:36, 141.77s/it]
⚠ Job 18605465 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_0_log.out
----------------------------------------
submitit INFO (2025-07-24 23:28:50,061) - Starting with JobEnvironment(job_id=18605465, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 23:28:50,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18605465/18605465_submitted.pkl
slurmstepd: error: *** JOB 18605465 ON n1719 CANCELLED AT 2025-07-25T01:28:58 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs : 35%|███░░░░░░░| 174/500 [15:19:28<39:37:55, 437.65s/it]
⚠ Job 18606032 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:07:19,074) - Starting with JobEnvironment(job_id=18606032, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:07:19,089) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606032/18606032_submitted.pkl
slurmstepd: error: *** JOB 18606032 ON n1719 CANCELLED AT 2025-07-25T05:07:42 DUE TO TIME LIMIT ***
⚠ Job 18606022 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606022, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606022/18606022_submitted.pkl
slurmstepd: error: *** JOB 18606022 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606015 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:03:33,497) - Starting with JobEnvironment(job_id=18606015, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:03:33,499) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606015/18606015_submitted.pkl
slurmstepd: error: *** JOB 18606015 ON n1720 CANCELLED AT 2025-07-25T05:03:42 DUE TO TIME LIMIT ***
⚠ Job 18606002 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:13,938) - Starting with JobEnvironment(job_id=18606002, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:13,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606002/18606002_submitted.pkl
slurmstepd: error: *** JOB 18606002 ON n1719 CANCELLED AT 2025-07-25T04:58:42 DUE TO TIME LIMIT ***
⚠ Job 18606013 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:01:23,231) - Starting with JobEnvironment(job_id=18606013, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:01:23,234) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606013/18606013_submitted.pkl
slurmstepd: error: *** JOB 18606013 ON n1719 CANCELLED AT 2025-07-25T05:01:41 DUE TO TIME LIMIT ***
⚠ Job 18606010 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,932) - Starting with JobEnvironment(job_id=18606010, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,936) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606010/18606010_submitted.pkl
slurmstepd: error: *** JOB 18606010 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606024 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:48,200) - Starting with JobEnvironment(job_id=18606024, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:48,204) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606024/18606024_submitted.pkl
slurmstepd: error: *** JOB 18606024 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606016 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:03:33,366) - Starting with JobEnvironment(job_id=18606016, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:03:33,370) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606016/18606016_submitted.pkl
slurmstepd: error: *** JOB 18606016 ON n1719 CANCELLED AT 2025-07-25T05:03:42 DUE TO TIME LIMIT ***
⚠ Job 18606027 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606027, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606027/18606027_submitted.pkl
slurmstepd: error: *** JOB 18606027 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606028 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,363) - Starting with JobEnvironment(job_id=18606028, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,365) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606028/18606028_submitted.pkl
slurmstepd: error: *** JOB 18606028 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606023 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,335) - Starting with JobEnvironment(job_id=18606023, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606023/18606023_submitted.pkl
slurmstepd: error: *** JOB 18606023 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606021 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,334) - Starting with JobEnvironment(job_id=18606021, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606021/18606021_submitted.pkl
slurmstepd: error: *** JOB 18606021 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606004 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,314) - Starting with JobEnvironment(job_id=18606004, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,316) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606004/18606004_submitted.pkl
slurmstepd: error: *** JOB 18606004 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606005 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:45,313) - Starting with JobEnvironment(job_id=18606005, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:45,316) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606005/18606005_submitted.pkl
slurmstepd: error: *** JOB 18606005 ON n1719 CANCELLED AT 2025-07-25T04:59:11 DUE TO TIME LIMIT ***
⚠ Job 18606029 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,336) - Starting with JobEnvironment(job_id=18606029, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,337) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606029/18606029_submitted.pkl
slurmstepd: error: *** JOB 18606029 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
⚠ Job 18606014 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:01:55,667) - Starting with JobEnvironment(job_id=18606014, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:01:55,669) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606014/18606014_submitted.pkl
slurmstepd: error: *** JOB 18606014 ON n1719 CANCELLED AT 2025-07-25T05:02:11 DUE TO TIME LIMIT ***
⚠ Job 18606003 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_0_log.out
----------------------------------------
submitit INFO (2025-07-25 02:58:13,938) - Starting with JobEnvironment(job_id=18606003, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 02:58:13,941) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606003/18606003_submitted.pkl
slurmstepd: error: *** JOB 18606003 ON n1719 CANCELLED AT 2025-07-25T04:58:42 DUE TO TIME LIMIT ***
⚠ Job 18606020 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:06:47,335) - Starting with JobEnvironment(job_id=18606020, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:06:47,336) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606020/18606020_submitted.pkl
slurmstepd: error: *** JOB 18606020 ON n1719 CANCELLED AT 2025-07-25T05:07:12 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job : 35%|███░░░░░░░| 174/500 [15:34:40<39:37:55, 437.65s/it]
⚠ Job 18606073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_0_log.out
----------------------------------------
submitit INFO (2025-07-25 03:09:57,479) - Starting with JobEnvironment(job_id=18606073, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 03:09:57,483) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606073/18606073_submitted.pkl
slurmstepd: error: *** JOB 18606073 ON n1719 CANCELLED AT 2025-07-25T05:10:13 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 114 ('VAL_ACC: ' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs : 40%|████░░░░░░| 201/500 [18:54:07<48:20:22, 582.01s/it]
⚠ Job 18606604 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,181) - Starting with JobEnvironment(job_id=18606604, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606604/18606604_submitted.pkl
slurmstepd: error: *** JOB 18606604 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606612 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:42,559) - Starting with JobEnvironment(job_id=18606612, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:42,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606612/18606612_submitted.pkl
slurmstepd: error: *** JOB 18606612 ON n1719 CANCELLED AT 2025-07-25T08:30:49 DUE TO TIME LIMIT ***
⚠ Job 18606613 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:42,554) - Starting with JobEnvironment(job_id=18606613, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:42,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606613/18606613_submitted.pkl
slurmstepd: error: *** JOB 18606613 ON n1719 CANCELLED AT 2025-07-25T08:30:49 DUE TO TIME LIMIT ***
⚠ Job 18606610 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:12,681) - Starting with JobEnvironment(job_id=18606610, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:12,683) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606610/18606610_submitted.pkl
slurmstepd: error: *** JOB 18606610 ON n1719 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606601 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,181) - Starting with JobEnvironment(job_id=18606601, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606601/18606601_submitted.pkl
slurmstepd: error: *** JOB 18606601 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
⚠ Job 18606603 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:30:13,180) - Starting with JobEnvironment(job_id=18606603, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:30:13,183) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606603/18606603_submitted.pkl
slurmstepd: error: *** JOB 18606603 ON n1720 CANCELLED AT 2025-07-25T08:30:19 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 120 ('VAL_ACC: ' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs : 40%|████░░░░░░| 201/500 [19:02:04<48:20:22, 582.01s/it]
⚠ Job 18606634 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:18,944) - Starting with JobEnvironment(job_id=18606634, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:18,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606634/18606634_submitted.pkl
slurmstepd: error: *** JOB 18606634 ON n1706 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606618 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,875) - Starting with JobEnvironment(job_id=18606618, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,878) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606618/18606618_submitted.pkl
slurmstepd: error: *** JOB 18606618 ON n1720 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606623 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606623, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606623/18606623_submitted.pkl
slurmstepd: error: *** JOB 18606623 ON n1707 CANCELLED AT 2025-07-25T08:38:50 DUE TO TIME LIMIT ***
⚠ Job 18606624 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606624, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606624/18606624_submitted.pkl
slurmstepd: error: *** JOB 18606624 ON n1707 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606636 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:18,944) - Starting with JobEnvironment(job_id=18606636, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:18,946) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606636/18606636_submitted.pkl
slurmstepd: error: *** JOB 18606636 ON n1706 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606627 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,501) - Starting with JobEnvironment(job_id=18606627, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,503) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606627/18606627_submitted.pkl
slurmstepd: error: *** JOB 18606627 ON n1707 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
⚠ Job 18606632 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:43:19,357) - Starting with JobEnvironment(job_id=18606632, hostname=n1707, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:43:19,359) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606632/18606632_submitted.pkl
slurmstepd: error: *** JOB 18606632 ON n1707 CANCELLED AT 2025-07-25T08:43:19 DUE TO TIME LIMIT ***
⚠ Job 18606620 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:38:46,265) - Starting with JobEnvironment(job_id=18606620, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:38:46,266) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606620/18606620_submitted.pkl
slurmstepd: error: *** JOB 18606620 ON n1719 CANCELLED AT 2025-07-25T08:38:49 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 128 ('VAL_ACC: ' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs : 40%|████░░░░░░| 201/500 [19:12:21<48:20:22, 582.01s/it]
⚠ Job 18606669 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:55:11,886) - Starting with JobEnvironment(job_id=18606669, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:55:11,890) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606669/18606669_submitted.pkl
slurmstepd: error: *** JOB 18606669 ON n1696 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
⚠ Job 18606667 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,951) - Starting with JobEnvironment(job_id=18606667, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606667/18606667_submitted.pkl
slurmstepd: error: *** JOB 18606667 ON n1706 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
⚠ Job 18606679 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,375) - Starting with JobEnvironment(job_id=18606679, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,378) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606679/18606679_submitted.pkl
slurmstepd: error: *** JOB 18606679 ON n1696 CANCELLED AT 2025-07-25T08:52:19 DUE TO TIME LIMIT ***
⚠ Job 18606663 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:56:16,950) - Starting with JobEnvironment(job_id=18606663, hostname=n1706, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:56:16,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606663/18606663_submitted.pkl
slurmstepd: error: *** JOB 18606663 ON n1706 CANCELLED AT 2025-07-25T08:48:19 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 132 ('VAL_ACC: ' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs : 40%|████░░░░░░| 201/500 [19:21:46<48:20:22, 582.01s/it]
⚠ Job 18606686 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606686, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606686/18606686_submitted.pkl
slurmstepd: error: *** JOB 18606686 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606687 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606687, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606687/18606687_submitted.pkl
slurmstepd: error: *** JOB 18606687 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606685 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:25,873) - Starting with JobEnvironment(job_id=18606685, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:25,875) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606685/18606685_submitted.pkl
slurmstepd: error: *** JOB 18606685 ON n1695 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
⚠ Job 18606682 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_0_log.out
----------------------------------------
submitit INFO (2025-07-25 06:58:24,915) - Starting with JobEnvironment(job_id=18606682, hostname=n1696, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 06:58:24,916) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18606682/18606682_submitted.pkl
slurmstepd: error: *** JOB 18606682 ON n1696 CANCELLED AT 2025-07-25T08:58:49 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 136 ('VAL_ACC: ' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs : 41%|████░░░░░░| 207/500 [22:34:31<101:33:48, 1247.88s/it]
⚠ Job 18608232 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608232, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608232/18608232_submitted.pkl
slurmstepd: error: *** JOB 18608232 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608230 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608230, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608230/18608230_submitted.pkl
slurmstepd: error: *** JOB 18608230 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608233 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,337) - Starting with JobEnvironment(job_id=18608233, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,340) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608233/18608233_submitted.pkl
slurmstepd: error: *** JOB 18608233 ON n1720 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608224 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608224, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608224/18608224_submitted.pkl
slurmstepd: error: *** JOB 18608224 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608227 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608227, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608227/18608227_submitted.pkl
slurmstepd: error: *** JOB 18608227 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608226 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608226, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608226/18608226_submitted.pkl
slurmstepd: error: *** JOB 18608226 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608231 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:55,917) - Starting with JobEnvironment(job_id=18608231, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:55,918) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608231/18608231_submitted.pkl
slurmstepd: error: *** JOB 18608231 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608235 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608235, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608235/18608235_submitted.pkl
slurmstepd: error: *** JOB 18608235 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608228 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608228, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608228/18608228_submitted.pkl
slurmstepd: error: *** JOB 18608228 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608225 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608225, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608225/18608225_submitted.pkl
slurmstepd: error: *** JOB 18608225 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608234 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,300) - Starting with JobEnvironment(job_id=18608234, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,302) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608234/18608234_submitted.pkl
slurmstepd: error: *** JOB 18608234 ON n1720 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608237 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608237, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608237/18608237_submitted.pkl
slurmstepd: error: *** JOB 18608237 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
⚠ Job 18608229 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608229, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608229/18608229_submitted.pkl
slurmstepd: error: *** JOB 18608229 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608223 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:30:51,077) - Starting with JobEnvironment(job_id=18608223, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:30:51,078) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608223/18608223_submitted.pkl
slurmstepd: error: *** JOB 18608223 ON n1720 CANCELLED AT 2025-07-25T12:30:53 DUE TO TIME LIMIT ***
⚠ Job 18608236 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:31:19,302) - Starting with JobEnvironment(job_id=18608236, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:31:19,304) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608236/18608236_submitted.pkl
slurmstepd: error: *** JOB 18608236 ON n1719 CANCELLED AT 2025-07-25T12:31:24 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 151 ('VAL_ACC: ' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs : 42%|████░░░░░░| 208/500 [23:04:03<86:06:01, 1061.51s/it]
⚠ Job 18608380 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:47:55,018) - Starting with JobEnvironment(job_id=18608380, hostname=n1691, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:47:55,019) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608380/18608380_submitted.pkl
slurmstepd: error: *** JOB 18608380 ON n1691 CANCELLED AT 2025-07-25T12:48:24 DUE TO TIME LIMIT ***
⚠ Job 18608448 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:51:02,547) - Starting with JobEnvironment(job_id=18608448, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:51:02,549) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608448/18608448_submitted.pkl
slurmstepd: error: *** JOB 18608448 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608443 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608443, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608443/18608443_submitted.pkl
slurmstepd: error: *** JOB 18608443 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608446 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608446, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608446/18608446_submitted.pkl
slurmstepd: error: *** JOB 18608446 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608397 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,158) - Starting with JobEnvironment(job_id=18608397, hostname=n1691, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,161) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608397/18608397_submitted.pkl
slurmstepd: error: *** JOB 18608397 ON n1691 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608450 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608450, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608450/18608450_submitted.pkl
slurmstepd: error: *** JOB 18608450 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608395 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,557) - Starting with JobEnvironment(job_id=18608395, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,559) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608395/18608395_submitted.pkl
slurmstepd: error: *** JOB 18608395 ON n1695 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608444 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608444, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608444/18608444_submitted.pkl
slurmstepd: error: *** JOB 18608444 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608464 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:51:25,674) - Starting with JobEnvironment(job_id=18608464, hostname=n1629, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:51:25,676) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608464/18608464_submitted.pkl
slurmstepd: error: *** JOB 18608464 ON n1629 CANCELLED AT 2025-07-25T12:51:54 DUE TO TIME LIMIT ***
⚠ Job 18608394 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:48:55,558) - Starting with JobEnvironment(job_id=18608394, hostname=n1695, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:48:55,560) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608394/18608394_submitted.pkl
slurmstepd: error: *** JOB 18608394 ON n1695 CANCELLED AT 2025-07-25T12:49:23 DUE TO TIME LIMIT ***
⚠ Job 18608453 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608453, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608453/18608453_submitted.pkl
slurmstepd: error: *** JOB 18608453 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608451 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608451, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608451/18608451_submitted.pkl
slurmstepd: error: *** JOB 18608451 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
⚠ Job 18608452 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:50:56,653) - Starting with JobEnvironment(job_id=18608452, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:50:56,655) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608452/18608452_submitted.pkl
slurmstepd: error: *** JOB 18608452 ON n1687 CANCELLED AT 2025-07-25T12:51:24 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 164 ('VAL_ACC: ' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 2 jobs : 42%|████░░░░░░| 208/500 [23:22:19<86:06:01, 1061.51s/it]
⚠ Job 18608545 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:56:26,046) - Starting with JobEnvironment(job_id=18608545, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:56:26,048) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608545/18608545_submitted.pkl
slurmstepd: error: *** JOB 18608545 ON n1687 CANCELLED AT 2025-07-25T12:56:54 DUE TO TIME LIMIT ***
⚠ Job 18608525 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_0_log.out
----------------------------------------
submitit INFO (2025-07-25 10:54:55,941) - Starting with JobEnvironment(job_id=18608525, hostname=n1687, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 10:54:55,942) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18608525/18608525_submitted.pkl
slurmstepd: error: *** JOB 18608525 ON n1687 CANCELLED AT 2025-07-25T12:54:54 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 167 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs : 43%|████░░░░░░| 213/500 [27:11:34<134:24:39, 1685.99s/it]
⚠ Job 18612963 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:54,720) - Starting with JobEnvironment(job_id=18612963, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:54,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612963/18612963_submitted.pkl
slurmstepd: error: *** JOB 18612963 ON n1570 CANCELLED AT 2025-07-25T16:57:06 DUE TO TIME LIMIT ***
⚠ Job 18612983 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,908) - Starting with JobEnvironment(job_id=18612983, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,910) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612983/18612983_submitted.pkl
slurmstepd: error: *** JOB 18612983 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612978 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,938) - Starting with JobEnvironment(job_id=18612978, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612978/18612978_submitted.pkl
slurmstepd: error: *** JOB 18612978 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612937 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:55:54,431) - Starting with JobEnvironment(job_id=18612937, hostname=n1624, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:55:54,435) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612937/18612937_submitted.pkl
slurmstepd: error: *** JOB 18612937 ON n1624 CANCELLED AT 2025-07-25T16:56:05 DUE TO TIME LIMIT ***
⚠ Job 18612981 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,905) - Starting with JobEnvironment(job_id=18612981, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,907) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612981/18612981_submitted.pkl
slurmstepd: error: *** JOB 18612981 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612977 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,941) - Starting with JobEnvironment(job_id=18612977, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612977/18612977_submitted.pkl
slurmstepd: error: *** JOB 18612977 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612976 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,828) - Starting with JobEnvironment(job_id=18612976, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,831) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612976/18612976_submitted.pkl
slurmstepd: error: *** JOB 18612976 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612950 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:24,010) - Starting with JobEnvironment(job_id=18612950, hostname=n1589, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:24,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612950/18612950_submitted.pkl
slurmstepd: error: *** JOB 18612950 ON n1589 CANCELLED AT 2025-07-25T16:56:35 DUE TO TIME LIMIT ***
⚠ Job 18612979 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,937) - Starting with JobEnvironment(job_id=18612979, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,939) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612979/18612979_submitted.pkl
slurmstepd: error: *** JOB 18612979 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
⚠ Job 18612962 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:56:54,720) - Starting with JobEnvironment(job_id=18612962, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:56:54,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612962/18612962_submitted.pkl
slurmstepd: error: *** JOB 18612962 ON n1570 CANCELLED AT 2025-07-25T16:57:06 DUE TO TIME LIMIT ***
⚠ Job 18612864 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:53:52,971) - Starting with JobEnvironment(job_id=18612864, hostname=n1700, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:53:52,973) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612864/18612864_submitted.pkl
slurmstepd: error: *** JOB 18612864 ON n1700 CANCELLED AT 2025-07-25T16:54:05 DUE TO TIME LIMIT ***
⚠ Job 18612982 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:24,941) - Starting with JobEnvironment(job_id=18612982, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:24,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612982/18612982_submitted.pkl
slurmstepd: error: *** JOB 18612982 ON n1570 CANCELLED AT 2025-07-25T16:57:35 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 179 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs : 43%|████░░░░░░| 213/500 [27:24:41<134:24:39, 1685.99s/it]
⚠ Job 18613048 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_0_log.out
----------------------------------------
submitit INFO (2025-07-25 15:02:56,501) - Starting with JobEnvironment(job_id=18613048, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 15:02:56,504) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613048/18613048_submitted.pkl
slurmstepd: error: *** JOB 18613048 ON n1719 CANCELLED AT 2025-07-25T17:03:07 DUE TO TIME LIMIT ***
⚠ Job 18613052 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_0_log.out
----------------------------------------
submitit INFO (2025-07-25 15:03:27,938) - Starting with JobEnvironment(job_id=18613052, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 15:03:27,941) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18613052/18613052_submitted.pkl
slurmstepd: error: *** JOB 18613052 ON n1719 CANCELLED AT 2025-07-25T17:03:38 DUE TO TIME LIMIT ***
⚠ Job 18612985 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_0_log.out
----------------------------------------
submitit INFO (2025-07-25 14:57:55,193) - Starting with JobEnvironment(job_id=18612985, hostname=n1570, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 14:57:55,195) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18612985/18612985_submitted.pkl
slurmstepd: error: *** JOB 18612985 ON n1570 CANCELLED AT 2025-07-25T16:58:05 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 183 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs : 43%|████░░░░░░| 214/500 [31:03:56<415:50:52, 5234.45s/it]
⚠ Job 18614252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,376) - Starting with JobEnvironment(job_id=18614252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614252/18614252_submitted.pkl
slurmstepd: error: *** JOB 18614252 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:00:28,118) - Starting with JobEnvironment(job_id=18614245, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:00:28,121) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614245/18614245_submitted.pkl
slurmstepd: error: *** JOB 18614245 ON n1718 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614239, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614239/18614239_submitted.pkl
slurmstepd: error: *** JOB 18614239 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614251/18614251_submitted.pkl
slurmstepd: error: *** JOB 18614251 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614246/18614246_submitted.pkl
slurmstepd: error: *** JOB 18614246 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614253 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614253, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:34,016) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614253/18614253_submitted.pkl
slurmstepd: error: *** JOB 18614253 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614243, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614243/18614243_submitted.pkl
slurmstepd: error: *** JOB 18614243 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614248/18614248_submitted.pkl
slurmstepd: error: *** JOB 18614248 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614250/18614250_submitted.pkl
slurmstepd: error: *** JOB 18614250 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614244, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614244/18614244_submitted.pkl
slurmstepd: error: *** JOB 18614244 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614238 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614238, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,037) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614238/18614238_submitted.pkl
slurmstepd: error: *** JOB 18614238 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614240, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614240/18614240_submitted.pkl
slurmstepd: error: *** JOB 18614240 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,993) - Starting with JobEnvironment(job_id=18614247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614247/18614247_submitted.pkl
slurmstepd: error: *** JOB 18614247 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614241, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,038) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614241/18614241_submitted.pkl
slurmstepd: error: *** JOB 18614241 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
⚠ Job 18614249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:01:33,994) - Starting with JobEnvironment(job_id=18614249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:01:33,996) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614249/18614249_submitted.pkl
slurmstepd: error: *** JOB 18614249 ON n1720 CANCELLED AT 2025-07-25T20:52:14 DUE TO TIME LIMIT ***
⚠ Job 18614242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:09:57,036) - Starting with JobEnvironment(job_id=18614242, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:09:57,037) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614242/18614242_submitted.pkl
slurmstepd: error: *** JOB 18614242 ON n1719 CANCELLED AT 2025-07-25T20:51:44 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 199 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs : 43%|████░░░░░░| 214/500 [31:28:29<415:50:52, 5234.45s/it]
⚠ Job 18614462 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:15:50,380) - Starting with JobEnvironment(job_id=18614462, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:15:50,382) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614462/18614462_submitted.pkl
slurmstepd: error: *** JOB 18614462 ON n1718 CANCELLED AT 2025-07-25T21:16:15 DUE TO TIME LIMIT ***
⚠ Job 18614463 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:21,556) - Starting with JobEnvironment(job_id=18614463, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:21,576) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614463/18614463_submitted.pkl
slurmstepd: error: *** JOB 18614463 ON n1718 CANCELLED AT 2025-07-25T21:16:45 DUE TO TIME LIMIT ***
⚠ Job 18614461 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:15:50,349) - Starting with JobEnvironment(job_id=18614461, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:15:50,365) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614461/18614461_submitted.pkl
slurmstepd: error: *** JOB 18614461 ON n1718 CANCELLED AT 2025-07-25T21:16:15 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 202 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs : 43%|████░░░░░░| 214/500 [31:41:17<415:50:52, 5234.45s/it]
⚠ Job 18614472 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614472, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614472/18614472_submitted.pkl
slurmstepd: error: *** JOB 18614472 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614467 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:51,734) - Starting with JobEnvironment(job_id=18614467, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:51,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614467/18614467_submitted.pkl
slurmstepd: error: *** JOB 18614467 ON n1717 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18614474 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614474, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614474/18614474_submitted.pkl
slurmstepd: error: *** JOB 18614474 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614470 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,493) - Starting with JobEnvironment(job_id=18614470, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,495) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614470/18614470_submitted.pkl
slurmstepd: error: *** JOB 18614470 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614468 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,346) - Starting with JobEnvironment(job_id=18614468, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,348) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614468/18614468_submitted.pkl
slurmstepd: error: *** JOB 18614468 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614489 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:23:56,341) - Starting with JobEnvironment(job_id=18614489, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:23:56,344) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614489/18614489_submitted.pkl
slurmstepd: error: *** JOB 18614489 ON n1716 CANCELLED AT 2025-07-25T21:24:15 DUE TO TIME LIMIT ***
⚠ Job 18614475 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614475, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614475/18614475_submitted.pkl
slurmstepd: error: *** JOB 18614475 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614465 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:50,973) - Starting with JobEnvironment(job_id=18614465, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:50,975) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614465/18614465_submitted.pkl
slurmstepd: error: *** JOB 18614465 ON n1718 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18614490 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:23:56,341) - Starting with JobEnvironment(job_id=18614490, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:23:56,344) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614490/18614490_submitted.pkl
slurmstepd: error: *** JOB 18614490 ON n1716 CANCELLED AT 2025-07-25T21:24:15 DUE TO TIME LIMIT ***
⚠ Job 18614473 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614473, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614473/18614473_submitted.pkl
slurmstepd: error: *** JOB 18614473 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614471 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:22,077) - Starting with JobEnvironment(job_id=18614471, hostname=n1716, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:22,079) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614471/18614471_submitted.pkl
slurmstepd: error: *** JOB 18614471 ON n1716 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
⚠ Job 18614466 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:16:50,973) - Starting with JobEnvironment(job_id=18614466, hostname=n1718, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:16:50,975) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614466/18614466_submitted.pkl
slurmstepd: error: *** JOB 18614466 ON n1718 CANCELLED AT 2025-07-25T21:17:15 DUE TO TIME LIMIT ***
⚠ Job 18614469 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_0_log.out
----------------------------------------
submitit INFO (2025-07-25 19:17:21,346) - Starting with JobEnvironment(job_id=18614469, hostname=n1717, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 19:17:21,348) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18614469/18614469_submitted.pkl
slurmstepd: error: *** JOB 18614469 ON n1717 CANCELLED AT 2025-07-25T21:17:45 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 217 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs : 43%|████░░░░░░| 215/500 [34:34:00<584:56:34, 7388.75s/it]
⚠ Job 18615250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,976) - Starting with JobEnvironment(job_id=18615250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615250/18615250_submitted.pkl
slurmstepd: error: *** JOB 18615250 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615254 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,758) - Starting with JobEnvironment(job_id=18615254, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615254/18615254_submitted.pkl
slurmstepd: error: *** JOB 18615254 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,896) - Starting with JobEnvironment(job_id=18615249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,898) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615249/18615249_submitted.pkl
slurmstepd: error: *** JOB 18615249 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,852) - Starting with JobEnvironment(job_id=18615242, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,854) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615242/18615242_submitted.pkl
slurmstepd: error: *** JOB 18615242 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:22,352) - Starting with JobEnvironment(job_id=18615240, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:22,355) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615240/18615240_submitted.pkl
slurmstepd: error: *** JOB 18615240 ON n1720 CANCELLED AT 2025-07-26T01:19:27 DUE TO TIME LIMIT ***
⚠ Job 18615243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,852) - Starting with JobEnvironment(job_id=18615243, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,854) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615243/18615243_submitted.pkl
slurmstepd: error: *** JOB 18615243 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,757) - Starting with JobEnvironment(job_id=18615251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615251/18615251_submitted.pkl
slurmstepd: error: *** JOB 18615251 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:53,791) - Starting with JobEnvironment(job_id=18615245, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:53,793) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615245/18615245_submitted.pkl
slurmstepd: error: *** JOB 18615245 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615253 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,757) - Starting with JobEnvironment(job_id=18615253, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615253/18615253_submitted.pkl
slurmstepd: error: *** JOB 18615253 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
⚠ Job 18615248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,903) - Starting with JobEnvironment(job_id=18615248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,905) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615248/18615248_submitted.pkl
slurmstepd: error: *** JOB 18615248 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,864) - Starting with JobEnvironment(job_id=18615241, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,867) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615241/18615241_submitted.pkl
slurmstepd: error: *** JOB 18615241 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,896) - Starting with JobEnvironment(job_id=18615246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,899) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615246/18615246_submitted.pkl
slurmstepd: error: *** JOB 18615246 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:52,976) - Starting with JobEnvironment(job_id=18615247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:52,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615247/18615247_submitted.pkl
slurmstepd: error: *** JOB 18615247 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:53,790) - Starting with JobEnvironment(job_id=18615244, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:53,793) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615244/18615244_submitted.pkl
slurmstepd: error: *** JOB 18615244 ON n1720 CANCELLED AT 2025-07-26T01:19:57 DUE TO TIME LIMIT ***
⚠ Job 18615239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:19:22,352) - Starting with JobEnvironment(job_id=18615239, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:19:22,355) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615239/18615239_submitted.pkl
slurmstepd: error: *** JOB 18615239 ON n1720 CANCELLED AT 2025-07-26T01:19:27 DUE TO TIME LIMIT ***
⚠ Job 18615252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:20:23,758) - Starting with JobEnvironment(job_id=18615252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:20:23,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615252/18615252_submitted.pkl
slurmstepd: error: *** JOB 18615252 ON n1720 CANCELLED AT 2025-07-26T01:20:27 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 233 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs : 43%|████░░░░░░| 217/500 [36:14:42<392:42:28, 4995.58s/it]
⚠ Job 18615323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:51:36,163) - Starting with JobEnvironment(job_id=18615323, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:51:36,165) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615323/18615323_submitted.pkl
slurmstepd: error: *** JOB 18615323 ON n1719 CANCELLED AT 2025-07-26T01:51:57 DUE TO TIME LIMIT ***
⚠ Job 18615322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:51:05,763) - Starting with JobEnvironment(job_id=18615322, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:51:05,765) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615322/18615322_submitted.pkl
slurmstepd: error: *** JOB 18615322 ON n1719 CANCELLED AT 2025-07-26T01:51:27 DUE TO TIME LIMIT ***
⚠ Job 18615320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:50:40,191) - Starting with JobEnvironment(job_id=18615320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:50:40,194) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615320/18615320_submitted.pkl
slurmstepd: error: *** JOB 18615320 ON n1720 CANCELLED AT 2025-07-26T01:50:58 DUE TO TIME LIMIT ***
⚠ Job 18615319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:50:35,268) - Starting with JobEnvironment(job_id=18615319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:50:35,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615319/18615319_submitted.pkl
slurmstepd: error: *** JOB 18615319 ON n1720 CANCELLED AT 2025-07-26T01:50:58 DUE TO TIME LIMIT ***
⚠ Job 18615325 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_0_log.out
----------------------------------------
submitit INFO (2025-07-25 23:52:07,037) - Starting with JobEnvironment(job_id=18615325, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-25 23:52:07,041) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18615325/18615325_submitted.pkl
slurmstepd: error: *** JOB 18615325 ON n1719 CANCELLED AT 2025-07-26T01:52:27 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 238 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs : 44%|████░░░░░░| 219/500 [39:53:26<420:38:00, 5388.90s/it]
⚠ Job 18616077 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616077, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616077/18616077_submitted.pkl
slurmstepd: error: *** JOB 18616077 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616078 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616078, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616078/18616078_submitted.pkl
slurmstepd: error: *** JOB 18616078 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616080 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,012) - Starting with JobEnvironment(job_id=18616080, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616080/18616080_submitted.pkl
slurmstepd: error: *** JOB 18616080 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616076 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616076, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616076/18616076_submitted.pkl
slurmstepd: error: *** JOB 18616076 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616081 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616081, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616081/18616081_submitted.pkl
slurmstepd: error: *** JOB 18616081 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616070 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616070, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616070/18616070_submitted.pkl
slurmstepd: error: *** JOB 18616070 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,012) - Starting with JobEnvironment(job_id=18616079, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,014) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616079/18616079_submitted.pkl
slurmstepd: error: *** JOB 18616079 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616069 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616069, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616069/18616069_submitted.pkl
slurmstepd: error: *** JOB 18616069 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616068, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616068/18616068_submitted.pkl
slurmstepd: error: *** JOB 18616068 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616074 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616074, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,894) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616074/18616074_submitted.pkl
slurmstepd: error: *** JOB 18616074 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616071 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616071, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616071/18616071_submitted.pkl
slurmstepd: error: *** JOB 18616071 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616072 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616072, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616072/18616072_submitted.pkl
slurmstepd: error: *** JOB 18616072 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616075, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616075/18616075_submitted.pkl
slurmstepd: error: *** JOB 18616075 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616083 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,010) - Starting with JobEnvironment(job_id=18616083, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,012) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616083/18616083_submitted.pkl
slurmstepd: error: *** JOB 18616083 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
⚠ Job 18616073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:08,893) - Starting with JobEnvironment(job_id=18616073, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:08,895) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616073/18616073_submitted.pkl
slurmstepd: error: *** JOB 18616073 ON n1720 CANCELLED AT 2025-07-26T06:08:32 DUE TO TIME LIMIT ***
⚠ Job 18616082 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:08:39,011) - Starting with JobEnvironment(job_id=18616082, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:08:39,013) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616082/18616082_submitted.pkl
slurmstepd: error: *** JOB 18616082 ON n1720 CANCELLED AT 2025-07-26T06:09:01 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 254 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs : 44%|████░░░░░░| 219/500 [40:41:35<420:38:00, 5388.90s/it]
⚠ Job 18616157 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,733) - Starting with JobEnvironment(job_id=18616157, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616157/18616157_submitted.pkl
slurmstepd: error: *** JOB 18616157 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616146 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:19:46,566) - Starting with JobEnvironment(job_id=18616146, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:19:46,569) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616146/18616146_submitted.pkl
slurmstepd: error: *** JOB 18616146 ON n1720 CANCELLED AT 2025-07-26T06:20:03 DUE TO TIME LIMIT ***
⚠ Job 18616155 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,227) - Starting with JobEnvironment(job_id=18616155, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,229) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616155/18616155_submitted.pkl
slurmstepd: error: *** JOB 18616155 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616159 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,732) - Starting with JobEnvironment(job_id=18616159, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616159/18616159_submitted.pkl
slurmstepd: error: *** JOB 18616159 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616150 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,206) - Starting with JobEnvironment(job_id=18616150, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616150/18616150_submitted.pkl
slurmstepd: error: *** JOB 18616150 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616147 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:20:47,074) - Starting with JobEnvironment(job_id=18616147, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:20:47,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616147/18616147_submitted.pkl
slurmstepd: error: *** JOB 18616147 ON n1720 CANCELLED AT 2025-07-26T06:21:03 DUE TO TIME LIMIT ***
⚠ Job 18616160 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,731) - Starting with JobEnvironment(job_id=18616160, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,734) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616160/18616160_submitted.pkl
slurmstepd: error: *** JOB 18616160 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616149 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:18,030) - Starting with JobEnvironment(job_id=18616149, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:18,032) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616149/18616149_submitted.pkl
slurmstepd: error: *** JOB 18616149 ON n1720 CANCELLED AT 2025-07-26T06:22:33 DUE TO TIME LIMIT ***
⚠ Job 18616153 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,160) - Starting with JobEnvironment(job_id=18616153, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,174) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616153/18616153_submitted.pkl
slurmstepd: error: *** JOB 18616153 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616156 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,206) - Starting with JobEnvironment(job_id=18616156, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616156/18616156_submitted.pkl
slurmstepd: error: *** JOB 18616156 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616148 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:20:47,074) - Starting with JobEnvironment(job_id=18616148, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:20:47,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616148/18616148_submitted.pkl
slurmstepd: error: *** JOB 18616148 ON n1720 CANCELLED AT 2025-07-26T06:21:03 DUE TO TIME LIMIT ***
⚠ Job 18616158 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:23:18,733) - Starting with JobEnvironment(job_id=18616158, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:23:18,735) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616158/18616158_submitted.pkl
slurmstepd: error: *** JOB 18616158 ON n1720 CANCELLED AT 2025-07-26T06:23:34 DUE TO TIME LIMIT ***
⚠ Job 18616151 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,161) - Starting with JobEnvironment(job_id=18616151, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,163) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616151/18616151_submitted.pkl
slurmstepd: error: *** JOB 18616151 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616154 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,205) - Starting with JobEnvironment(job_id=18616154, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,207) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616154/18616154_submitted.pkl
slurmstepd: error: *** JOB 18616154 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
⚠ Job 18616152 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:22:48,202) - Starting with JobEnvironment(job_id=18616152, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:22:48,220) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616152/18616152_submitted.pkl
slurmstepd: error: *** JOB 18616152 ON n1720 CANCELLED AT 2025-07-26T06:23:03 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 269 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs : 44%|████░░░░░░| 219/500 [41:00:01<420:38:00, 5388.90s/it]
⚠ Job 18616165 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:36:58,253) - Starting with JobEnvironment(job_id=18616165, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:36:58,256) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616165/18616165_submitted.pkl
slurmstepd: error: *** JOB 18616165 ON n1720 CANCELLED AT 2025-07-26T06:37:03 DUE TO TIME LIMIT ***
⚠ Job 18616162 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:32:25,166) - Starting with JobEnvironment(job_id=18616162, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:32:25,180) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616162/18616162_submitted.pkl
slurmstepd: error: *** JOB 18616162 ON n1720 CANCELLED AT 2025-07-26T06:32:33 DUE TO TIME LIMIT ***
⚠ Job 18616171 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,818) - Starting with JobEnvironment(job_id=18616171, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,837) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616171/18616171_submitted.pkl
slurmstepd: error: *** JOB 18616171 ON n1720 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616169 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,297) - Starting with JobEnvironment(job_id=18616169, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,309) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616169/18616169_submitted.pkl
slurmstepd: error: *** JOB 18616169 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616170 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,338) - Starting with JobEnvironment(job_id=18616170, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,350) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616170/18616170_submitted.pkl
slurmstepd: error: *** JOB 18616170 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616176 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:38:28,647) - Starting with JobEnvironment(job_id=18616176, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:38:28,650) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616176/18616176_submitted.pkl
slurmstepd: error: *** JOB 18616176 ON n1719 CANCELLED AT 2025-07-26T06:38:33 DUE TO TIME LIMIT ***
⚠ Job 18616161 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:30:54,138) - Starting with JobEnvironment(job_id=18616161, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:30:54,140) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616161/18616161_submitted.pkl
slurmstepd: error: *** JOB 18616161 ON n1720 CANCELLED AT 2025-07-26T06:31:03 DUE TO TIME LIMIT ***
⚠ Job 18616167 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,294) - Starting with JobEnvironment(job_id=18616167, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,297) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616167/18616167_submitted.pkl
slurmstepd: error: *** JOB 18616167 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616172 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:59,210) - Starting with JobEnvironment(job_id=18616172, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:59,214) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616172/18616172_submitted.pkl
slurmstepd: error: *** JOB 18616172 ON n1720 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616174 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,756) - Starting with JobEnvironment(job_id=18616174, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616174/18616174_submitted.pkl
slurmstepd: error: *** JOB 18616174 ON n1719 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
⚠ Job 18616168 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:28,295) - Starting with JobEnvironment(job_id=18616168, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:28,319) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616168/18616168_submitted.pkl
slurmstepd: error: *** JOB 18616168 ON n1720 CANCELLED AT 2025-07-26T06:37:33 DUE TO TIME LIMIT ***
⚠ Job 18616175 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:38:28,648) - Starting with JobEnvironment(job_id=18616175, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:38:28,651) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616175/18616175_submitted.pkl
slurmstepd: error: *** JOB 18616175 ON n1719 CANCELLED AT 2025-07-26T06:38:33 DUE TO TIME LIMIT ***
⚠ Job 18616163 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:33:25,757) - Starting with JobEnvironment(job_id=18616163, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:33:25,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616163/18616163_submitted.pkl
slurmstepd: error: *** JOB 18616163 ON n1720 CANCELLED AT 2025-07-26T06:33:33 DUE TO TIME LIMIT ***
⚠ Job 18616173 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:37:58,756) - Starting with JobEnvironment(job_id=18616173, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:37:58,759) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616173/18616173_submitted.pkl
slurmstepd: error: *** JOB 18616173 ON n1719 CANCELLED AT 2025-07-26T06:38:03 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 283 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 2 jobs : 44%|████░░░░░░| 219/500 [41:09:51<420:38:00, 5388.90s/it]
⚠ Job 18616178 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:45:03,317) - Starting with JobEnvironment(job_id=18616178, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:45:03,320) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616178/18616178_submitted.pkl
slurmstepd: error: *** JOB 18616178 ON n1719 CANCELLED AT 2025-07-26T06:45:03 DUE TO TIME LIMIT ***
⚠ Job 18616177 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_0_log.out
----------------------------------------
submitit INFO (2025-07-26 04:44:02,655) - Starting with JobEnvironment(job_id=18616177, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-26 04:44:02,657) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18616177/18616177_submitted.pkl
slurmstepd: error: *** JOB 18616177 ON n1719 CANCELLED AT 2025-07-26T06:44:03 DUE TO TIME LIMIT ***
BoTorchGenerator, failed: 285 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, waiting for 2 jobs, finished 2 jobs : 44%|████░░░░░░| 219/500 [41:14:51<52:55:29, 678.04s/it]
Best VAL_ACC, max (total: 219, failed: 285)
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃OO_Info_SLURM_JOB_ID┃epochs┃lr ┃batch_size┃hidden_size┃dropout┃num_dense_layers┃weight_decay┃activation┃init ┃VAL_ACC┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│18612975.0 │200 │0.0001│1409 │6875 │0.5 │1 │0.0 │leaky_relu│normal│98.47 │
└──────────────────────┴────────┴────────┴────────────┴─────────────┴─────────┴──────────────────┴──────────────┴────────────┴────────┴─────────┘
Runtime Infos
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃Number of evaluations┃ Min time ┃ Max time ┃ Average time ┃ Median time ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ 224 │ 30.00 sec │ 7144.00 sec (1h59m4s) │ 1700.08 sec (28m20s) │ 968.00 sec (16m8s) │
└───────────────────────┴────────────┴───────────────────────┴──────────────────────┴────────────────────┘
2025-07-24 13:38:24: SOBOL, Started OmniOpt2 run...
2025-07-24 13:38:34: Sobol, getting new HP set #1/50
2025-07-24 13:38:43: Sobol, getting new HP set #2/50
2025-07-24 13:38:51: Sobol, getting new HP set #3/50
2025-07-24 13:38:59: Sobol, getting new HP set #4/50
2025-07-24 13:39:07: Sobol, getting new HP set #5/50
2025-07-24 13:39:16: Sobol, getting new HP set #6/50
2025-07-24 13:39:24: Sobol, getting new HP set #7/50
2025-07-24 13:39:32: Sobol, getting new HP set #8/50
2025-07-24 13:39:40: Sobol, getting new HP set #9/50
2025-07-24 13:39:49: Sobol, getting new HP set #10/50
2025-07-24 13:39:57: Sobol, getting new HP set #11/50
2025-07-24 13:40:05: Sobol, getting new HP set #12/50
2025-07-24 13:40:14: Sobol, getting new HP set #13/50
2025-07-24 13:40:22: Sobol, getting new HP set #14/50
2025-07-24 13:40:30: Sobol, getting new HP set #15/50
2025-07-24 13:40:38: Sobol, getting new HP set #16/50
2025-07-24 13:40:46: Sobol, getting new HP set #17/50
2025-07-24 13:40:55: Sobol, getting new HP set #18/50
2025-07-24 13:41:03: Sobol, getting new HP set #19/50
2025-07-24 13:41:12: Sobol, getting new HP set #20/50
2025-07-24 13:41:20: Sobol, getting new HP set #21/50
2025-07-24 13:41:28: Sobol, getting new HP set #22/50
2025-07-24 13:41:36: Sobol, getting new HP set #23/50
2025-07-24 13:41:45: Sobol, getting new HP set #24/50
2025-07-24 13:41:53: Sobol, getting new HP set #25/50
2025-07-24 13:42:01: Sobol, getting new HP set #26/50
2025-07-24 13:42:11: Sobol, getting new HP set #27/50
2025-07-24 13:42:19: Sobol, getting new HP set #28/50
2025-07-24 13:42:29: Sobol, getting new HP set #29/50
2025-07-24 13:42:38: Sobol, getting new HP set #30/50
2025-07-24 13:42:46: Sobol, getting new HP set #31/50
2025-07-24 13:42:55: Sobol, getting new HP set #32/50
2025-07-24 13:43:03: Sobol, getting new HP set #33/50
2025-07-24 13:43:12: Sobol, getting new HP set #34/50
2025-07-24 13:43:20: Sobol, getting new HP set #35/50
2025-07-24 13:43:28: Sobol, getting new HP set #36/50
2025-07-24 13:43:37: Sobol, getting new HP set #37/50
2025-07-24 13:43:45: Sobol, getting new HP set #38/50
2025-07-24 13:43:53: Sobol, getting new HP set #39/50
2025-07-24 13:44:01: Sobol, getting new HP set #40/50
2025-07-24 13:44:10: Sobol, getting new HP set #41/50
2025-07-24 13:44:18: Sobol, getting new HP set #42/50
2025-07-24 13:44:27: Sobol, getting new HP set #43/50
2025-07-24 13:44:35: Sobol, getting new HP set #44/50
2025-07-24 13:44:43: Sobol, getting new HP set #45/50
2025-07-24 13:44:52: Sobol, getting new HP set #46/50
2025-07-24 13:45:00: Sobol, getting new HP set #47/50
2025-07-24 13:45:09: Sobol, getting new HP set #48/50
2025-07-24 13:45:17: Sobol, getting new HP set #49/50
2025-07-24 13:45:26: Sobol, getting new HP set #50/50
2025-07-24 13:45:35: Sobol, requested 50 jobs, got 50, 8.40 s/job
2025-07-24 13:45:39: Sobol, eval #1/50 start
2025-07-24 13:45:45: Sobol, eval #2/50 start
2025-07-24 13:45:49: Sobol, eval #3/50 start
2025-07-24 13:45:54: Sobol, eval #4/50 start
2025-07-24 13:45:58: Sobol, eval #5/50 start
2025-07-24 13:46:03: Sobol, eval #6/50 start
2025-07-24 13:46:07: Sobol, eval #7/50 start
2025-07-24 13:46:12: Sobol, eval #8/50 start
2025-07-24 13:46:17: Sobol, eval #9/50 start
2025-07-24 13:46:22: Sobol, eval #10/50 start
2025-07-24 13:46:27: Sobol, eval #11/50 start
2025-07-24 13:46:31: Sobol, eval #12/50 start
2025-07-24 13:46:36: Sobol, eval #13/50 start
2025-07-24 13:46:41: Sobol, eval #14/50 start
2025-07-24 13:46:45: Sobol, eval #15/50 start
2025-07-24 13:46:50: Sobol, eval #16/50 start
2025-07-24 13:46:54: Sobol, eval #17/50 start
2025-07-24 13:46:58: Sobol, eval #18/50 start
2025-07-24 13:47:03: Sobol, eval #19/50 start
2025-07-24 13:47:07: Sobol, eval #20/50 start
2025-07-24 13:47:12: Sobol, eval #21/50 start
2025-07-24 13:47:16: Sobol, eval #22/50 start
2025-07-24 13:47:21: Sobol, eval #23/50 start
2025-07-24 13:47:26: Sobol, eval #24/50 start
2025-07-24 13:47:30: Sobol, eval #25/50 start
2025-07-24 13:47:34: Sobol, eval #26/50 start
2025-07-24 13:47:39: Sobol, eval #27/50 start
2025-07-24 13:47:43: Sobol, eval #28/50 start
2025-07-24 13:47:48: Sobol, eval #29/50 start
2025-07-24 13:47:54: Sobol, eval #30/50 start
2025-07-24 13:47:59: Sobol, eval #31/50 start
2025-07-24 13:48:03: Sobol, eval #32/50 start
2025-07-24 13:48:08: Sobol, eval #33/50 start
2025-07-24 13:48:12: Sobol, eval #34/50 start
2025-07-24 13:48:17: Sobol, eval #35/50 start
2025-07-24 13:48:21: Sobol, eval #36/50 start
2025-07-24 13:48:26: Sobol, eval #37/50 start
2025-07-24 13:48:31: Sobol, eval #38/50 start
2025-07-24 13:48:35: Sobol, eval #39/50 start
2025-07-24 13:48:39: Sobol, eval #40/50 start
2025-07-24 13:48:44: Sobol, eval #41/50 start
2025-07-24 13:48:48: Sobol, eval #42/50 start
2025-07-24 13:48:52: Sobol, eval #43/50 start
2025-07-24 13:48:57: Sobol, eval #44/50 start
2025-07-24 13:49:01: Sobol, eval #45/50 start
2025-07-24 13:49:05: Sobol, eval #46/50 start
2025-07-24 13:49:10: Sobol, eval #47/50 start
2025-07-24 13:49:14: Sobol, eval #48/50 start
2025-07-24 13:49:18: Sobol, eval #49/50 start
2025-07-24 13:49:23: Sobol, eval #50/50 start
2025-07-24 13:49:28: Sobol, starting new job
2025-07-24 13:49:28: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:28: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:29: Sobol, starting new job
2025-07-24 13:49:56: Sobol, unknown 8 = ∑8/50, started new job
2025-07-24 13:49:56: Sobol, unknown 8 = ∑8/50, started new job
2025-07-24 13:49:57: Sobol, unknown 8 = ∑8/50, started new job
2025-07-24 13:49:57: Sobol, unknown 8 = ∑8/50, started new job
2025-07-24 13:49:57: Sobol, unknown 8 = ∑8/50, started new job
2025-07-24 13:49:57: Sobol, unknown 10 = ∑10/50, started new job
2025-07-24 13:49:58: Sobol, unknown 12 = ∑12/50, started new job
2025-07-24 13:49:59: Sobol, unknown 12 = ∑12/50, started new job
2025-07-24 13:50:00: Sobol, unknown 16 = ∑16/50, started new job
2025-07-24 13:50:00: Sobol, unknown 16 = ∑16/50, started new job
2025-07-24 13:50:00: Sobol, unknown 16 = ∑16/50, started new job
2025-07-24 13:50:01: Sobol, unknown 16 = ∑16/50, started new job
2025-07-24 13:50:01: Sobol, unknown 16 = ∑16/50, started new job
2025-07-24 13:50:04: Sobol, pending 16 = ∑16/50, started new job
2025-07-24 13:50:04: Sobol, pending 16 = ∑16/50, started new job
2025-07-24 13:50:04: Sobol, pending 16 = ∑16/50, started new job
2025-07-24 13:50:33: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-24 13:50:33: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-24 13:50:36: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-24 13:50:37: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-24 13:50:41: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-24 13:50:47: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:48: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:48: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:48: Sobol, running/unknown 16/1 = ∑17/50, started new job
2025-07-24 13:50:49: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:49: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:49: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:49: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:50: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:50: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:51: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-24 13:50:55: Sobol, running/pending 16/2 = ∑18/50, started new job
2025-07-24 13:51:04: Sobol, running/pending/unknown 16/2/2 = ∑20/50, started new job
2025-07-24 13:51:05: Sobol, running/pending/unknown 16/2/2 = ∑20/50, started new job
2025-07-24 13:51:18: Sobol, running/unknown 20/1 = ∑21/50, started new job
2025-07-24 13:51:28: Sobol, running/pending/unknown 20/1/4 = ∑25/50, started new job
2025-07-24 13:51:29: Sobol, running/pending/unknown 20/1/4 = ∑25/50, started new job
2025-07-24 13:51:29: Sobol, running/pending/unknown 20/1/4 = ∑25/50, started new job
2025-07-24 13:51:29: Sobol, running/pending/unknown 20/1/4 = ∑25/50, started new job
2025-07-24 13:51:32: Sobol, running/pending 20/6 = ∑26/50, started new job
2025-07-24 13:51:34: Sobol, running/pending/unknown 20/6/3 = ∑29/50, started new job
2025-07-24 13:51:34: Sobol, running/pending/unknown 20/6/3 = ∑29/50, started new job
2025-07-24 13:51:34: Sobol, running/pending/unknown 20/6/3 = ∑29/50, starting new job
2025-07-24 13:51:37: Sobol, running/pending/unknown 20/6/5 = ∑31/50, started new job
2025-07-24 13:51:39: Sobol, running/pending 20/11 = ∑31/50, started new job
2025-07-24 13:51:39: Sobol, running/pending 20/11 = ∑31/50, started new job
2025-07-24 13:51:42: Sobol, running/pending 20/11 = ∑31/50, starting new job
2025-07-24 13:51:42: Sobol, running/pending 20/11 = ∑31/50, starting new job
2025-07-24 13:51:42: Sobol, running/pending 20/11 = ∑31/50, starting new job
2025-07-24 13:51:42: Sobol, running/pending 20/11 = ∑31/50, starting new job
2025-07-24 13:51:57: Sobol, running/pending 30/1 = ∑31/50, starting new job
2025-07-24 13:52:20: Sobol, running 31 = ∑31/50, starting new job
2025-07-24 13:52:20: Sobol, running 31 = ∑31/50, starting new job
2025-07-24 13:52:20: Sobol, running 31 = ∑31/50, starting new job
2025-07-24 13:52:20: Sobol, running 31 = ∑31/50, starting new job
2025-07-24 13:52:22: Sobol, running/unknown 31/1 = ∑32/50, starting new job
2025-07-24 13:52:22: Sobol, running/unknown 31/1 = ∑32/50, started new job
2025-07-24 13:52:25: Sobol, running/pending 31/1 = ∑32/50, starting new job
2025-07-24 13:52:30: Sobol, running/pending/unknown 31/2/3 = ∑36/50, started new job
2025-07-24 13:52:30: Sobol, running/pending/unknown 31/2/3 = ∑36/50, started new job
2025-07-24 13:52:30: Sobol, running/pending/unknown 31/2/3 = ∑36/50, starting new job
2025-07-24 13:52:30: Sobol, running/pending/unknown 31/2/3 = ∑36/50, starting new job
2025-07-24 13:52:31: Sobol, running/pending/unknown 31/2/3 = ∑36/50, started new job
2025-07-24 13:52:31: Sobol, running/pending/unknown 31/2/3 = ∑36/50, started new job
2025-07-24 13:52:31: Sobol, running/pending/unknown 31/2/3 = ∑36/50, starting new job
2025-07-24 13:52:32: Sobol, running/pending 31/5 = ∑36/50, starting new job
2025-07-24 13:52:41: Sobol, running/pending/unknown 31/5/1 = ∑37/50, started new job
2025-07-24 13:53:10: Sobol, running/unknown 37/3 = ∑40/50, started new job
2025-07-24 13:53:11: Sobol, running/unknown 37/4 = ∑41/50, started new job
2025-07-24 13:53:11: Sobol, running/unknown 37/4 = ∑41/50, started new job
2025-07-24 13:53:13: Sobol, running/unknown 37/5 = ∑42/50, started new job
2025-07-24 13:53:14: Sobol, running/pending 37/5 = ∑42/50, started new job
2025-07-24 13:53:14: Sobol, running/pending 37/5 = ∑42/50, starting new job
2025-07-24 13:53:19: Sobol, running/pending/unknown 37/5/1 = ∑43/50, starting new job
2025-07-24 13:53:20: Sobol, running/pending 42/1 = ∑43/50, started new job
2025-07-24 13:53:23: Sobol, running/unknown 43/4 = ∑47/50, started new job
2025-07-24 13:53:23: Sobol, running/unknown 43/4 = ∑47/50, started new job
2025-07-24 13:53:25: Sobol, running/unknown 43/4 = ∑47/50, started new job
2025-07-24 13:53:25: Sobol, running/unknown 43/4 = ∑47/50, started new job
2025-07-24 13:53:26: Sobol, running/unknown 43/4 = ∑47/50, starting new job
2025-07-24 13:54:01: Sobol, running/unknown 47/1 = ∑48/50, started new job
2025-07-24 13:54:04: Sobol, running/pending/unknown 47/1/1 = ∑49/50, started new job
2025-07-24 13:54:08: Sobol, best VAL_ACC: 10.28, running/completed/pending/unknown 46/1/2/1 = ∑50/50, started new job
2025-07-24 13:54:20: Sobol, best VAL_ACC: 10.28, running/completed/pending 46/1/3 = ∑50/50, new result: 10.28
2025-07-24 13:54:43: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, finishing jobs, finished 1 job
2025-07-24 13:54:52: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:55:03: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:55:15: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:55:29: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:55:41: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:55:53: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:56:05: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:56:17: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 13:56:29: Sobol, best VAL_ACC: 10.28, running 49 = ∑49/50, new result: 0.75
2025-07-24 13:56:57: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-24 13:57:04: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:57:16: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:57:28: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:57:40: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:57:53: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:58:05: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:58:17: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:58:29: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:58:41: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:58:53: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:59:05: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:59:17: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:59:30: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:59:42: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 13:59:54: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 14:00:06: Sobol, best VAL_ACC: 10.28, running 48 = ∑48/50, new result: 0.08
2025-07-24 14:00:32: Sobol, best VAL_ACC: 10.28, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-24 14:00:39: Sobol, best VAL_ACC: 10.28, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 14:00:50: Sobol, best VAL_ACC: 10.28, running 47 = ∑47/50, new result: 11.36
2025-07-24 14:01:11: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-24 14:01:17: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:01:30: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:01:42: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:01:56: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:02:08: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:02:21: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:02:33: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:02:45: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:02:57: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:03:10: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 14:03:21: Sobol, best VAL_ACC: 11.36, running 46 = ∑46/50, new result: 10.09
2025-07-24 14:03:41: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job
2025-07-24 14:03:49: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:04:01: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:04:13: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:04:25: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:04:37: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:04:49: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:05:02: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:05:14: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:05:26: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:05:38: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:05:51: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:06:03: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:06:15: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:06:28: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:06:40: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:06:53: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:07:05: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:07:17: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:07:30: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:07:42: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:07:56: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:08:08: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:08:21: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:08:34: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:08:46: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:08:58: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:09:10: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:09:22: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:09:34: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:09:47: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:10:00: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:10:13: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:10:25: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:10:37: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:10:50: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:11:02: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:11:15: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:11:27: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:11:39: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:11:52: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:12:04: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:12:17: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:12:29: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 14:12:41: Sobol, best VAL_ACC: 11.36, running 45 = ∑45/50, new result: 10.28
2025-07-24 14:13:07: Sobol, best VAL_ACC: 11.36, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job
2025-07-24 14:13:14: Sobol, best VAL_ACC: 11.36, running 44 = ∑44/50, waiting for 44 jobs
2025-07-24 14:13:25: Sobol, best VAL_ACC: 11.36, running 44 = ∑44/50, new result: 0.41
2025-07-24 14:13:45: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 44 jobs, finished 1 job
2025-07-24 14:13:52: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:14:03: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:14:16: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:14:28: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:14:40: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:14:52: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:15:06: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 14:15:18: Sobol, best VAL_ACC: 11.36, running 43 = ∑43/50, new result: 10.28
2025-07-24 14:15:39: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 43 jobs, finished 1 job
2025-07-24 14:15:46: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:15:58: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:16:10: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:16:22: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:16:35: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:16:47: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:16:59: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:17:11: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:17:24: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:17:36: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:17:49: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:18:01: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:18:13: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:18:26: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:18:38: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:18:50: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:19:03: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 14:19:14: Sobol, best VAL_ACC: 11.36, running 42 = ∑42/50, new result: 24.85
2025-07-24 14:19:36: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 42 jobs, finished 1 job
2025-07-24 14:19:43: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:19:55: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:20:07: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:20:19: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:20:33: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:20:45: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:20:59: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:21:12: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 14:21:24: Sobol, best VAL_ACC: 24.85, running 41 = ∑41/50, new result: 11.35
2025-07-24 14:21:45: Sobol, best VAL_ACC: 24.85, running 40 = ∑40/50, waiting for 41 jobs, finished 1 job
2025-07-24 14:21:51: Sobol, best VAL_ACC: 24.85, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 14:22:05: Sobol, best VAL_ACC: 24.85, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 14:22:17: Sobol, best VAL_ACC: 24.85, running 40 = ∑40/50, new result: 5.12
2025-07-24 14:22:38: Sobol, best VAL_ACC: 24.85, running 39 = ∑39/50, waiting for 40 jobs, finished 1 job
2025-07-24 14:22:45: Sobol, best VAL_ACC: 24.85, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 14:22:56: Sobol, best VAL_ACC: 24.85, running 39 = ∑39/50, new result: 10.28
2025-07-24 14:23:17: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job
2025-07-24 14:23:23: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 14:23:35: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 14:23:47: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 14:24:00: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 14:24:14: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 14:24:25: Sobol, best VAL_ACC: 24.85, running 38 = ∑38/50, new result: 9.58
2025-07-24 14:24:46: Sobol, best VAL_ACC: 24.85, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job
2025-07-24 14:24:52: Sobol, best VAL_ACC: 24.85, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 14:25:05: Sobol, best VAL_ACC: 24.85, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 14:25:16: Sobol, best VAL_ACC: 24.85, running 37 = ∑37/50, new result: 10.28
2025-07-24 14:25:38: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job
2025-07-24 14:25:51: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:26:05: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:26:17: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:26:29: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:26:42: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:26:54: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:27:06: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:27:18: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:27:31: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:27:43: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:27:56: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:28:09: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:28:21: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:28:34: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:28:46: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:28:59: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:29:11: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:29:23: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:29:36: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:29:48: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:30:02: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:30:14: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:30:29: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:30:56: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:31:09: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:31:21: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:31:33: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:31:45: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:31:58: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:32:10: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:32:22: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 14:32:34: Sobol, best VAL_ACC: 24.85, running 36 = ∑36/50, new result: 31.3
2025-07-24 14:33:17: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job
2025-07-24 14:33:23: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:33:36: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:33:48: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:34:00: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:34:13: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:34:25: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 14:34:37: Sobol, best VAL_ACC: 31.3, running 35 = ∑35/50, new result: 9.17
2025-07-24 14:34:58: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 35 jobs, finished 1 job
2025-07-24 14:35:05: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:35:17: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:35:29: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:35:42: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:35:55: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:36:10: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:36:26: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:36:57: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:37:09: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:37:22: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:37:34: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:37:48: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:38:01: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:38:13: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:38:26: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:38:38: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:38:50: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:39:03: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:39:16: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:39:29: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:39:41: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:39:53: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:40:05: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:40:18: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:40:30: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:40:43: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:40:55: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:41:07: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:41:20: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:41:32: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:41:44: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:41:57: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:42:11: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:42:23: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:42:35: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:43:09: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:43:21: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:43:34: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 14:43:56: Sobol, best VAL_ACC: 31.3, running 34 = ∑34/50, new result: 11.35
2025-07-24 14:44:21: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job
2025-07-24 14:44:29: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:44:59: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:45:14: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:45:27: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:45:39: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:45:59: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:46:13: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:46:26: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:46:38: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:46:57: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:47:11: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:47:36: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:47:54: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:48:10: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:48:38: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:48:54: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:49:11: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:49:28: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:49:43: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:49:56: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:50:09: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:50:21: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:50:49: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:51:01: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:51:14: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:51:27: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:51:41: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:51:53: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:52:05: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:52:19: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:52:31: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:52:43: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:52:55: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:53:08: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:53:20: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:53:34: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:53:46: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:53:58: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:54:11: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:54:23: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:54:35: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:54:48: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:55:00: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:55:12: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:55:25: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:55:37: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:55:49: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:56:01: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:56:14: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:56:26: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:56:38: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:56:51: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:57:03: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:57:16: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:57:28: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:57:40: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:57:54: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:58:07: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:58:19: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:58:32: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:58:44: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:58:57: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:59:09: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:59:22: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:59:41: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 14:59:53: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:00:06: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:00:20: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:00:33: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:00:46: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:00:58: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:01:11: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:01:23: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:01:36: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:01:49: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:02:03: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:02:26: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:02:55: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:03:08: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:03:20: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:03:33: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:03:45: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:03:58: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:04:10: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:04:24: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:04:37: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:04:49: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:05:02: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:05:14: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:05:26: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:05:39: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:05:51: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:06:03: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:06:16: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:06:28: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:06:41: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:06:53: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:07:07: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:07:19: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:07:32: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:07:45: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:07:58: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:08:12: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:08:26: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:08:53: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:09:06: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:09:19: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:09:36: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 15:09:59: Sobol, best VAL_ACC: 31.3, running 33 = ∑33/50, new result: 11.35
2025-07-24 15:10:22: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job
2025-07-24 15:10:28: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:10:41: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:10:55: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:11:07: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:11:20: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:11:42: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:11:56: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:12:09: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:12:22: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:12:35: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:12:49: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:13:01: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:13:15: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:13:27: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:13:40: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:13:54: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:14:07: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:14:20: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:14:32: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:14:46: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:14:58: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:15:12: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:15:25: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:15:37: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:15:50: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:16:03: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:16:15: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:16:27: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:16:59: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:17:12: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:17:26: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:18:00: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:18:12: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:18:25: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:18:42: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:18:57: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:19:09: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:19:22: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:19:36: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:20:00: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:20:12: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:20:25: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:20:37: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:20:53: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:21:06: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:21:18: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:21:33: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:21:47: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:21:59: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:22:11: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:22:34: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:22:52: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:23:05: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:23:18: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:23:53: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:24:06: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:24:23: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:24:40: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:24:53: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:25:05: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:25:21: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:25:33: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:25:46: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:26:00: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:26:14: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:26:27: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:26:40: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:26:55: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:27:08: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:27:20: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:27:33: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:27:47: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:28:00: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:28:12: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:28:36: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:29:00: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:29:25: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:29:42: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:29:55: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:30:08: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:30:21: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:30:33: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:30:46: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:30:59: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:31:11: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:31:25: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:31:38: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:31:51: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:32:05: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:32:29: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:32:42: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:32:55: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:33:08: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:33:21: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:33:33: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:33:46: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:34:14: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 15:34:27: Sobol, best VAL_ACC: 31.3, running 32 = ∑32/50, new result: 10.28
2025-07-24 15:34:49: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job
2025-07-24 15:34:59: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:35:11: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:35:25: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:35:37: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:35:50: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:36:05: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:36:17: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:36:30: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:36:43: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:36:57: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:37:09: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:37:22: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:37:35: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:37:48: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:38:00: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:38:13: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:38:28: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:38:42: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:38:55: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:39:08: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:39:21: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:39:34: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:39:46: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:40:00: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:40:13: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:40:25: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:40:39: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:40:52: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:41:05: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 15:41:17: Sobol, best VAL_ACC: 31.3, running 31 = ∑31/50, new result: 10.28
2025-07-24 15:41:48: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job
2025-07-24 15:41:55: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:42:10: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:42:23: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:42:35: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:42:48: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:43:01: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:43:15: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:43:33: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:43:54: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:44:07: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:44:20: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:44:33: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:44:46: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:44:59: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:45:11: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:45:28: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:45:40: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:45:54: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:46:07: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:46:20: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:46:32: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:46:45: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:46:57: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:47:10: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:47:23: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:47:35: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:47:48: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:48:00: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:48:12: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:48:25: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:48:37: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:48:50: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:49:09: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:49:21: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:49:34: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:49:47: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:49:59: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:50:12: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:50:25: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:50:39: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:50:51: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:51:03: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:51:16: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:51:28: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:51:40: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:51:54: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:52:08: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:52:20: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:52:33: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:52:45: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:52:58: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:53:11: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:53:23: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:53:37: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:53:49: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:54:02: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:54:15: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:54:28: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:54:40: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:54:53: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:55:06: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:55:18: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:55:31: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:55:43: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:55:56: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:56:09: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:56:23: Sobol, best VAL_ACC: 31.3, running 30 = ∑30/50, waiting for 30 jobs
2025-07-24 15:56:53: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:54: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:54: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:54: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:54: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:56: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:58: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:58: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:58: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:58: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:58: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:56:57: Sobol, best VAL_ACC: 31.3, timeout 30 = ∑30/50, job_failed
2025-07-24 15:59:24: Sobol, failed: 30, best VAL_ACC: 31.3, waiting for 30 jobs, finished 30 jobs
2025-07-24 15:59:33: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #1/50
2025-07-24 15:59:48: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #2/50
2025-07-24 16:00:01: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #3/50
2025-07-24 16:00:21: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #4/50
2025-07-24 16:00:35: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #5/50
2025-07-24 16:01:09: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #6/50
2025-07-24 16:01:24: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #7/50
2025-07-24 16:01:38: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #8/50
2025-07-24 16:01:52: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #9/50
2025-07-24 16:02:28: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #10/50
2025-07-24 16:02:53: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #11/50
2025-07-24 16:03:07: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #12/50
2025-07-24 16:03:21: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #13/50
2025-07-24 16:03:34: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #14/50
2025-07-24 16:03:48: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #15/50
2025-07-24 16:04:02: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #16/50
2025-07-24 16:04:15: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #17/50
2025-07-24 16:04:30: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #18/50
2025-07-24 16:04:43: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #19/50
2025-07-24 16:04:57: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #20/50
2025-07-24 16:05:11: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #21/50
2025-07-24 16:05:25: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #22/50
2025-07-24 16:05:39: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #23/50
2025-07-24 16:05:53: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #24/50
2025-07-24 16:06:07: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #25/50
2025-07-24 16:06:20: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #26/50
2025-07-24 16:06:34: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #27/50
2025-07-24 16:06:48: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #28/50
2025-07-24 16:07:03: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #29/50
2025-07-24 16:07:16: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #30/50
2025-07-24 16:07:31: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #31/50
2025-07-24 16:07:45: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #32/50
2025-07-24 16:07:59: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #33/50
2025-07-24 16:08:12: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #34/50
2025-07-24 16:08:27: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #35/50
2025-07-24 16:08:41: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #36/50
2025-07-24 16:08:55: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #37/50
2025-07-24 16:09:09: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #38/50
2025-07-24 16:09:22: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #39/50
2025-07-24 16:09:36: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #40/50
2025-07-24 16:09:50: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #41/50
2025-07-24 16:10:04: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #42/50
2025-07-24 16:10:17: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #43/50
2025-07-24 16:10:31: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #44/50
2025-07-24 16:10:45: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #45/50
2025-07-24 16:10:58: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #46/50
2025-07-24 16:11:19: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #47/50
2025-07-24 16:11:32: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #48/50
2025-07-24 16:11:47: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #49/50
2025-07-24 16:12:00: Sobol, failed: 30, best VAL_ACC: 31.3, getting new HP set #50/50
2025-07-24 16:12:15: Sobol, failed: 30, best VAL_ACC: 31.3, requested 50 jobs, got 50, 15.26 s/job
2025-07-24 16:12:22: Sobol, failed: 30, best VAL_ACC: 31.3, eval #1/50 start
2025-07-24 16:12:30: Sobol, failed: 30, best VAL_ACC: 31.3, eval #2/50 start
2025-07-24 16:12:49: Sobol, failed: 30, best VAL_ACC: 31.3, eval #3/50 start
2025-07-24 16:12:58: Sobol, failed: 30, best VAL_ACC: 31.3, eval #4/50 start
2025-07-24 16:13:06: Sobol, failed: 30, best VAL_ACC: 31.3, eval #5/50 start
2025-07-24 16:13:14: Sobol, failed: 30, best VAL_ACC: 31.3, eval #6/50 start
2025-07-24 16:13:21: Sobol, failed: 30, best VAL_ACC: 31.3, eval #7/50 start
2025-07-24 16:13:28: Sobol, failed: 30, best VAL_ACC: 31.3, eval #8/50 start
2025-07-24 16:13:36: Sobol, failed: 30, best VAL_ACC: 31.3, eval #9/50 start
2025-07-24 16:14:01: Sobol, failed: 30, best VAL_ACC: 31.3, eval #10/50 start
2025-07-24 16:14:09: Sobol, failed: 30, best VAL_ACC: 31.3, eval #11/50 start
2025-07-24 16:14:18: Sobol, failed: 30, best VAL_ACC: 31.3, eval #12/50 start
2025-07-24 16:14:25: Sobol, failed: 30, best VAL_ACC: 31.3, eval #13/50 start
2025-07-24 16:14:33: Sobol, failed: 30, best VAL_ACC: 31.3, eval #14/50 start
2025-07-24 16:14:40: Sobol, failed: 30, best VAL_ACC: 31.3, eval #15/50 start
2025-07-24 16:14:48: Sobol, failed: 30, best VAL_ACC: 31.3, eval #16/50 start
2025-07-24 16:14:55: Sobol, failed: 30, best VAL_ACC: 31.3, eval #17/50 start
2025-07-24 16:15:03: Sobol, failed: 30, best VAL_ACC: 31.3, eval #18/50 start
2025-07-24 16:15:10: Sobol, failed: 30, best VAL_ACC: 31.3, eval #19/50 start
2025-07-24 16:15:17: Sobol, failed: 30, best VAL_ACC: 31.3, eval #20/50 start
2025-07-24 16:15:24: Sobol, failed: 30, best VAL_ACC: 31.3, eval #21/50 start
2025-07-24 16:15:32: Sobol, failed: 30, best VAL_ACC: 31.3, eval #22/50 start
2025-07-24 16:15:39: Sobol, failed: 30, best VAL_ACC: 31.3, eval #23/50 start
2025-07-24 16:15:48: Sobol, failed: 30, best VAL_ACC: 31.3, eval #24/50 start
2025-07-24 16:15:56: Sobol, failed: 30, best VAL_ACC: 31.3, eval #25/50 start
2025-07-24 16:16:03: Sobol, failed: 30, best VAL_ACC: 31.3, eval #26/50 start
2025-07-24 16:16:11: Sobol, failed: 30, best VAL_ACC: 31.3, eval #27/50 start
2025-07-24 16:16:18: Sobol, failed: 30, best VAL_ACC: 31.3, eval #28/50 start
2025-07-24 16:16:26: Sobol, failed: 30, best VAL_ACC: 31.3, eval #29/50 start
2025-07-24 16:16:33: Sobol, failed: 30, best VAL_ACC: 31.3, eval #30/50 start
2025-07-24 16:16:40: Sobol, failed: 30, best VAL_ACC: 31.3, eval #31/50 start
2025-07-24 16:16:50: Sobol, failed: 30, best VAL_ACC: 31.3, eval #32/50 start
2025-07-24 16:16:57: Sobol, failed: 30, best VAL_ACC: 31.3, eval #33/50 start
2025-07-24 16:17:05: Sobol, failed: 30, best VAL_ACC: 31.3, eval #34/50 start
2025-07-24 16:17:18: Sobol, failed: 30, best VAL_ACC: 31.3, eval #35/50 start
2025-07-24 16:17:32: Sobol, failed: 30, best VAL_ACC: 31.3, eval #36/50 start
2025-07-24 16:17:41: Sobol, failed: 30, best VAL_ACC: 31.3, eval #37/50 start
2025-07-24 16:17:48: Sobol, failed: 30, best VAL_ACC: 31.3, eval #38/50 start
2025-07-24 16:17:55: Sobol, failed: 30, best VAL_ACC: 31.3, eval #39/50 start
2025-07-24 16:18:03: Sobol, failed: 30, best VAL_ACC: 31.3, eval #40/50 start
2025-07-24 16:18:10: Sobol, failed: 30, best VAL_ACC: 31.3, eval #41/50 start
2025-07-24 16:18:18: Sobol, failed: 30, best VAL_ACC: 31.3, eval #42/50 start
2025-07-24 16:18:26: Sobol, failed: 30, best VAL_ACC: 31.3, eval #43/50 start
2025-07-24 16:18:33: Sobol, failed: 30, best VAL_ACC: 31.3, eval #44/50 start
2025-07-24 16:18:41: Sobol, failed: 30, best VAL_ACC: 31.3, eval #45/50 start
2025-07-24 16:18:49: Sobol, failed: 30, best VAL_ACC: 31.3, eval #46/50 start
2025-07-24 16:18:56: Sobol, failed: 30, best VAL_ACC: 31.3, eval #47/50 start
2025-07-24 16:19:03: Sobol, failed: 30, best VAL_ACC: 31.3, eval #48/50 start
2025-07-24 16:19:11: Sobol, failed: 30, best VAL_ACC: 31.3, eval #49/50 start
2025-07-24 16:19:18: Sobol, failed: 30, best VAL_ACC: 31.3, eval #50/50 start
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:28: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:27: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:28: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:28: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:28: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:19:29: Sobol, failed: 30, best VAL_ACC: 31.3, starting new job
2025-07-24 16:20:31: Sobol, failed: 30, best VAL_ACC: 31.3, unknown 1 = ∑1/50, started new job
2025-07-24 16:20:35: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 1/3 = ∑4/50, started new job
2025-07-24 16:20:35: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 1/3 = ∑4/50, started new job
2025-07-24 16:20:36: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 1/3 = ∑4/50, started new job
2025-07-24 16:20:39: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/8 = ∑13/50, started new job
2025-07-24 16:20:39: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/8 = ∑13/50, started new job
2025-07-24 16:20:40: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/9 = ∑14/50, started new job
2025-07-24 16:20:40: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/8 = ∑13/50, started new job
2025-07-24 16:20:40: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/10 = ∑15/50, started new job
2025-07-24 16:20:40: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/11 = ∑16/50, started new job
2025-07-24 16:20:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/11 = ∑16/50, started new job
2025-07-24 16:20:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 5/11 = ∑16/50, started new job
2025-07-24 16:20:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, started new job
2025-07-24 16:20:43: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, started new job
2025-07-24 16:20:43: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, started new job
2025-07-24 16:20:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, started new job
2025-07-24 16:21:02: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, starting new job
2025-07-24 16:21:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, starting new job
2025-07-24 16:21:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending 16 = ∑16/50, starting new job
2025-07-24 16:22:01: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, started new job
2025-07-24 16:22:01: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, started new job
2025-07-24 16:22:01: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:01: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, started new job
2025-07-24 16:22:03: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:04: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:04: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:04: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:05: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:05: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:05: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:05: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:22:06: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:23:07: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:23:12: Sobol, failed: 30, best VAL_ACC: 31.3, pending 19 = ∑19/50, starting new job
2025-07-24 16:23:13: Sobol, failed: 30, pending 19 = ∑19/50, starting new job
2025-07-24 16:23:16: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 19/1 = ∑20/50, started new job
2025-07-24 16:23:17: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/3 = ∑23/50, starting new job
2025-07-24 16:23:18: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/8 = ∑28/50, started new job
2025-07-24 16:23:19: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/9 = ∑29/50, started new job
2025-07-24 16:23:19: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/9 = ∑29/50, started new job
2025-07-24 16:23:20: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/9 = ∑29/50, started new job
2025-07-24 16:23:20: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/10 = ∑30/50, started new job
2025-07-24 16:23:20: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/9 = ∑29/50, started new job
2025-07-24 16:23:21: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/10 = ∑30/50, started new job
2025-07-24 16:23:21: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/10 = ∑30/50, started new job
2025-07-24 16:23:21: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 20/10 = ∑30/50, started new job
2025-07-24 16:23:22: Sobol, failed: 30, best VAL_ACC: 31.3, pending 30 = ∑30/50, starting new job
2025-07-24 16:23:22: Sobol, failed: 30, best VAL_ACC: 31.3, pending 30 = ∑30/50, started new job
2025-07-24 16:23:23: Sobol, failed: 30, best VAL_ACC: 31.3, pending 30 = ∑30/50, starting new job
2025-07-24 16:24:18: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 30/2 = ∑32/50, started new job
2025-07-24 16:24:19: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 30/2 = ∑32/50, started new job
2025-07-24 16:24:21: Sobol, failed: 30, best VAL_ACC: 31.3, pending 32 = ∑32/50, starting new job
2025-07-24 16:24:21: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 32/1 = ∑33/50, started new job
2025-07-24 16:24:37: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, started new job
2025-07-24 16:24:37: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, started new job
2025-07-24 16:24:40: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:24:46: Sobol, failed: 30, best VAL_ACC: 31.3, pending 35 = ∑35/50, starting new job
2025-07-24 16:25:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 35/1 = ∑36/50, started new job
2025-07-24 16:25:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 35/1 = ∑36/50, starting new job
2025-07-24 16:25:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 35/1 = ∑36/50, starting new job
2025-07-24 16:25:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 35/1 = ∑36/50, starting new job
2025-07-24 16:25:46: Sobol, failed: 30, best VAL_ACC: 31.3, pending 36 = ∑36/50, starting new job
2025-07-24 16:25:49: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 36/2 = ∑38/50, starting new job
2025-07-24 16:25:52: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 36/2 = ∑38/50, started new job
2025-07-24 16:25:53: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 36/2 = ∑38/50, started new job
2025-07-24 16:25:54: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:54: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:54: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:55: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:55: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:55: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:25:56: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 40/5 = ∑45/50, started new job
2025-07-24 16:26:39: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 45/1 = ∑46/50, started new job
2025-07-24 16:26:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 46/2 = ∑48/50, started new job
2025-07-24 16:26:42: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 46/2 = ∑48/50, started new job
2025-07-24 16:26:45: Sobol, failed: 30, best VAL_ACC: 31.3, pending/unknown 46/3 = ∑49/50, started new job
2025-07-24 16:26:51: Sobol, failed: 30, best VAL_ACC: 31.3, pending 50 = ∑50/50, started new job
2025-07-24 16:27:26: Sobol, failed: 30, best VAL_ACC: 31.3, pending 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:27:41: Sobol, failed: 30, best VAL_ACC: 31.3, pending 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:27:54: Sobol, failed: 30, best VAL_ACC: 31.3, pending 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:28:15: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:28:35: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:28:52: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:29:07: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:29:23: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:29:39: Sobol, failed: 30, best VAL_ACC: 31.3, running/pending 20/30 = ∑50/50, waiting for 50 jobs
2025-07-24 16:29:57: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:30:12: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:30:43: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:31:03: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:31:19: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:31:36: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, waiting for 50 jobs
2025-07-24 16:31:50: Sobol, failed: 30, best VAL_ACC: 31.3, running 50 = ∑50/50, new result: 10.28
2025-07-24 16:32:20: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job
2025-07-24 16:32:37: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 16:32:53: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 16:33:16: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 16:33:31: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 16:33:56: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 16:34:10: Sobol, failed: 30, best VAL_ACC: 31.3, running 49 = ∑49/50, new result: 9.8
2025-07-24 16:34:40: Sobol, failed: 30, best VAL_ACC: 31.3, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-24 16:34:50: Sobol, failed: 30, best VAL_ACC: 31.3, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 16:35:05: Sobol, failed: 30, best VAL_ACC: 31.3, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 16:35:21: Sobol, failed: 30, best VAL_ACC: 31.3, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 16:35:36: Sobol, failed: 30, best VAL_ACC: 31.3, running 48 = ∑48/50, new result: 9.58
2025-07-24 16:36:04: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-24 16:36:14: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:36:29: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:36:45: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:37:02: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:37:18: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:37:33: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:37:49: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:38:04: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:38:19: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:38:45: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:39:00: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 16:39:18: Sobol, failed: 30, best VAL_ACC: 31.3, running 47 = ∑47/50, new result: 48.07
2025-07-24 16:39:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-24 16:39:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 16:40:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 16:40:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 16:40:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 16:41:18: Sobol, failed: 30, best VAL_ACC: 48.07, running 46 = ∑46/50, new result: 10.26
2025-07-24 16:42:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job
2025-07-24 16:42:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:42:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:42:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:43:00: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:43:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:43:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:44:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:44:26: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:44:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:44:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 16:45:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 45 = ∑45/50, new result: 10.09
2025-07-24 16:45:46: Sobol, failed: 30, best VAL_ACC: 48.07, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job
2025-07-24 16:45:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 44 = ∑44/50, waiting for 44 jobs
2025-07-24 16:46:11: Sobol, failed: 30, best VAL_ACC: 48.07, running 44 = ∑44/50, new result: 0.36
2025-07-24 16:49:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 44 jobs, finished 1 job
2025-07-24 16:49:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:49:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:50:17: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:50:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:50:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:51:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, waiting for 43 jobs
2025-07-24 16:51:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 43 = ∑43/50, new result: 9.82
2025-07-24 16:51:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 43 jobs, finished 1 job
2025-07-24 16:52:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:52:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:52:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:52:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:53:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:53:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:53:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:53:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:54:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:54:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:54:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:55:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:55:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:55:50: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:56:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:56:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:56:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:56:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:57:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:57:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 16:57:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, new result: 9.8
2025-07-24 16:57:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 42 = ∑42/50, new result: 11.35
2025-07-24 16:58:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 42 jobs, finished 2 jobs
2025-07-24 16:58:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 16:59:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 16:59:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 16:59:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:00:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:00:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:00:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:01:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:01:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:02:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:03:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:03:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:03:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:03:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 17:04:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 40 = ∑40/50, new result: 10.09
2025-07-24 17:04:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 40 jobs, finished 1 job
2025-07-24 17:04:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:05:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:05:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:05:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:05:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:06:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:06:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:06:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:06:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:07:18: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 17:07:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 39 = ∑39/50, new result: 10.1
2025-07-24 17:08:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job
2025-07-24 17:08:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:08:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:08:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:09:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:09:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:09:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:09:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:10:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:10:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:10:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:11:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:11:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:11:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:11:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:12:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:12:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:12:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:12:54: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:13:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:13:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:13:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:13:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:14:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:14:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:14:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:15:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 17:15:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 38 = ∑38/50, new result: 11.35
2025-07-24 17:16:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job
2025-07-24 17:16:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:16:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:16:44: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:16:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:17:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:17:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:17:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:18:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:18:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:18:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 17:18:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 37 = ∑37/50, new result: 9.87
2025-07-24 17:19:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job
2025-07-24 17:19:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 17:19:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 17:20:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 17:20:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 17:20:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 36 = ∑36/50, new result: 3.49
2025-07-24 17:21:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job
2025-07-24 17:21:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 17:21:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 35 = ∑35/50, new result: 11.95
2025-07-24 17:21:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 35 jobs, finished 1 job
2025-07-24 17:22:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:22:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:22:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:22:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:23:09: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:23:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:23:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:23:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:24:11: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:24:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:24:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:24:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:25:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:25:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:25:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:26:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:26:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:26:46: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:27:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:27:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:27:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:27:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:28:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:28:18: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:28:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:28:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:29:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:29:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:29:35: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:29:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:30:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:30:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:30:37: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:30:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:31:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:31:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:31:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 17:32:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 34 = ∑34/50, new result: 0.16
2025-07-24 17:32:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job
2025-07-24 17:32:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:33:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:33:18: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:33:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:33:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:34:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:34:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:34:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:34:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:35:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:35:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:35:37: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:35:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:36:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:36:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:36:42: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:36:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 17:37:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 33 = ∑33/50, new result: 10.09
2025-07-24 17:37:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job
2025-07-24 17:37:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:38:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:38:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:38:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:38:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:39:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:39:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:39:42: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:39:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:40:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:40:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:40:44: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:40:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:41:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:41:31: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:41:46: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:42:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:42:18: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:42:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:42:50: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:43:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:43:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:43:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:43:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:44:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:44:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:44:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:44:54: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:45:09: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:45:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:45:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:45:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:46:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:46:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:46:44: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:47:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:47:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:47:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:47:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:48:11: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:48:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:48:42: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:48:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:49:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:49:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:49:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:50:03: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:50:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:50:35: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:50:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:51:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:51:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:51:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:51:54: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:52:09: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:52:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:52:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:52:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:53:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:53:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:53:42: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:53:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:54:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:54:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:54:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:55:00: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:55:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:55:31: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:55:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:56:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:56:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:56:35: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:56:52: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:57:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:57:24: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:57:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:57:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:58:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:58:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:58:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:59:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:59:17: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:59:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 17:59:50: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:00:07: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:00:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:00:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:00:54: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:01:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:01:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:01:44: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:02:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:02:17: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:02:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:02:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:03:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:03:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:03:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:03:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:04:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:04:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:04:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:04:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:05:15: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:05:33: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:05:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:06:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:06:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:06:38: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:06:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:07:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:07:24: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:07:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:07:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:08:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:08:26: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:08:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:08:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:09:13: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:09:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:09:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:10:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:10:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:10:31: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:10:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:11:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:11:23: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:11:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:11:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:12:11: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:12:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:12:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:12:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:13:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:13:30: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:13:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:14:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:14:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:14:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:14:48: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:15:04: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:15:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:15:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:15:49: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:16:05: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:16:20: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:16:35: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:16:50: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:17:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:17:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:17:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:17:51: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:18:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:18:22: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:18:37: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:18:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:19:08: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:19:26: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:19:41: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:19:56: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:20:11: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:20:26: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:20:42: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:20:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:21:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:21:27: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:21:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:21:59: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:22:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:22:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:22:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:23:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:23:16: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:23:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:23:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:24:02: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:24:19: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:24:34: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:24:50: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:25:06: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:25:21: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:25:36: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:25:53: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:26:09: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:26:24: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:26:39: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:26:55: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:27:10: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:27:25: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:27:40: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:27:57: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:28:12: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:28:28: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:28:43: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:28:58: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:29:14: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:29:29: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:29:45: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:30:01: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:30:17: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:30:32: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:30:47: Sobol, failed: 30, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 18:31:20: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:20: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:20: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:20: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:20: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:19: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:22: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:22: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:22: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:22: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:22: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:21: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:23: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:23: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:23: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:23: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:24: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:26: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:31:27: Sobol, failed: 30, best VAL_ACC: 48.07, timeout 32 = ∑32/50, job_failed
2025-07-24 18:34:53: Sobol, failed: 62, best VAL_ACC: 48.07, waiting for 32 jobs, finished 32 jobs
2025-07-24 18:35:06: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #1/50
2025-07-24 18:35:24: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #2/50
2025-07-24 18:35:42: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #3/50
2025-07-24 18:36:00: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #4/50
2025-07-24 18:36:19: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #5/50
2025-07-24 18:36:38: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #6/50
2025-07-24 18:36:57: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #7/50
2025-07-24 18:37:16: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #8/50
2025-07-24 18:37:34: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #9/50
2025-07-24 18:37:52: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #10/50
2025-07-24 18:38:11: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #11/50
2025-07-24 18:38:29: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #12/50
2025-07-24 18:38:47: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #13/50
2025-07-24 18:39:06: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #14/50
2025-07-24 18:39:24: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #15/50
2025-07-24 18:39:43: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #16/50
2025-07-24 18:40:01: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #17/50
2025-07-24 18:40:20: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #18/50
2025-07-24 18:40:38: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #19/50
2025-07-24 18:40:56: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #20/50
2025-07-24 18:41:14: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #21/50
2025-07-24 18:41:32: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #22/50
2025-07-24 18:41:50: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #23/50
2025-07-24 18:42:09: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #24/50
2025-07-24 18:42:27: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #25/50
2025-07-24 18:42:45: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #26/50
2025-07-24 18:43:04: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #27/50
2025-07-24 18:43:22: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #28/50
2025-07-24 18:43:40: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #29/50
2025-07-24 18:43:58: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #30/50
2025-07-24 18:44:19: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #31/50
2025-07-24 18:44:37: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #32/50
2025-07-24 18:44:56: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #33/50
2025-07-24 18:45:14: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #34/50
2025-07-24 18:45:34: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #35/50
2025-07-24 18:45:52: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #36/50
2025-07-24 18:46:11: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #37/50
2025-07-24 18:46:29: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #38/50
2025-07-24 18:46:49: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #39/50
2025-07-24 18:47:08: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #40/50
2025-07-24 18:47:27: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #41/50
2025-07-24 18:47:46: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #42/50
2025-07-24 18:48:04: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #43/50
2025-07-24 18:48:23: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #44/50
2025-07-24 18:48:42: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #45/50
2025-07-24 18:49:01: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #46/50
2025-07-24 18:49:19: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #47/50
2025-07-24 18:49:38: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #48/50
2025-07-24 18:49:56: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #49/50
2025-07-24 18:50:14: Sobol, failed: 62, best VAL_ACC: 48.07, getting new HP set #50/50
2025-07-24 18:50:33: Sobol, failed: 62, best VAL_ACC: 48.07, requested 50 jobs, got 50, 18.58 s/job
2025-07-24 18:50:41: Sobol, failed: 62, best VAL_ACC: 48.07, eval #1/50 start
2025-07-24 18:50:51: Sobol, failed: 62, best VAL_ACC: 48.07, eval #2/50 start
2025-07-24 18:51:01: Sobol, failed: 62, best VAL_ACC: 48.07, eval #3/50 start
2025-07-24 18:51:11: Sobol, failed: 62, best VAL_ACC: 48.07, eval #4/50 start
2025-07-24 18:51:22: Sobol, failed: 62, best VAL_ACC: 48.07, eval #5/50 start
2025-07-24 18:51:33: Sobol, failed: 62, best VAL_ACC: 48.07, eval #6/50 start
2025-07-24 18:51:43: Sobol, failed: 62, best VAL_ACC: 48.07, eval #7/50 start
2025-07-24 18:51:53: Sobol, failed: 62, best VAL_ACC: 48.07, eval #8/50 start
2025-07-24 18:52:02: Sobol, failed: 62, best VAL_ACC: 48.07, eval #9/50 start
2025-07-24 18:52:12: Sobol, failed: 62, best VAL_ACC: 48.07, eval #10/50 start
2025-07-24 18:52:22: Sobol, failed: 62, best VAL_ACC: 48.07, eval #11/50 start
2025-07-24 18:52:34: Sobol, failed: 62, best VAL_ACC: 48.07, eval #12/50 start
2025-07-24 18:53:32: Sobol, failed: 62, best VAL_ACC: 48.07, eval #13/50 start
2025-07-24 18:53:43: Sobol, failed: 62, best VAL_ACC: 48.07, eval #14/50 start
2025-07-24 18:53:53: Sobol, failed: 62, best VAL_ACC: 48.07, eval #15/50 start
2025-07-24 18:54:04: Sobol, failed: 62, best VAL_ACC: 48.07, eval #16/50 start
2025-07-24 18:54:14: Sobol, failed: 62, best VAL_ACC: 48.07, eval #17/50 start
2025-07-24 18:54:24: Sobol, failed: 62, best VAL_ACC: 48.07, eval #18/50 start
2025-07-24 18:54:34: Sobol, failed: 62, best VAL_ACC: 48.07, eval #19/50 start
2025-07-24 18:54:45: Sobol, failed: 62, best VAL_ACC: 48.07, eval #20/50 start
2025-07-24 18:54:54: Sobol, failed: 62, best VAL_ACC: 48.07, eval #21/50 start
2025-07-24 18:55:04: Sobol, failed: 62, best VAL_ACC: 48.07, eval #22/50 start
2025-07-24 18:55:14: Sobol, failed: 62, best VAL_ACC: 48.07, eval #23/50 start
2025-07-24 18:55:24: Sobol, failed: 62, best VAL_ACC: 48.07, eval #24/50 start
2025-07-24 18:55:35: Sobol, failed: 62, best VAL_ACC: 48.07, eval #25/50 start
2025-07-24 18:55:45: Sobol, failed: 62, best VAL_ACC: 48.07, eval #26/50 start
2025-07-24 18:55:55: Sobol, failed: 62, best VAL_ACC: 48.07, eval #27/50 start
2025-07-24 18:56:06: Sobol, failed: 62, best VAL_ACC: 48.07, eval #28/50 start
2025-07-24 18:56:15: Sobol, failed: 62, best VAL_ACC: 48.07, eval #29/50 start
2025-07-24 18:56:25: Sobol, failed: 62, best VAL_ACC: 48.07, eval #30/50 start
2025-07-24 18:56:35: Sobol, failed: 62, best VAL_ACC: 48.07, eval #31/50 start
2025-07-24 18:56:45: Sobol, failed: 62, best VAL_ACC: 48.07, eval #32/50 start
2025-07-24 18:56:55: Sobol, failed: 62, best VAL_ACC: 48.07, eval #33/50 start
2025-07-24 18:57:05: Sobol, failed: 62, best VAL_ACC: 48.07, eval #34/50 start
2025-07-24 18:57:15: Sobol, failed: 62, best VAL_ACC: 48.07, eval #35/50 start
2025-07-24 18:57:26: Sobol, failed: 62, best VAL_ACC: 48.07, eval #36/50 start
2025-07-24 18:57:36: Sobol, failed: 62, best VAL_ACC: 48.07, eval #37/50 start
2025-07-24 18:57:46: Sobol, failed: 62, best VAL_ACC: 48.07, eval #38/50 start
2025-07-24 18:57:56: Sobol, failed: 62, best VAL_ACC: 48.07, eval #39/50 start
2025-07-24 18:58:06: Sobol, failed: 62, best VAL_ACC: 48.07, eval #40/50 start
2025-07-24 18:58:16: Sobol, failed: 62, best VAL_ACC: 48.07, eval #41/50 start
2025-07-24 18:58:26: Sobol, failed: 62, best VAL_ACC: 48.07, eval #42/50 start
2025-07-24 18:58:36: Sobol, failed: 62, best VAL_ACC: 48.07, eval #43/50 start
2025-07-24 18:58:46: Sobol, failed: 62, best VAL_ACC: 48.07, eval #44/50 start
2025-07-24 18:58:55: Sobol, failed: 62, best VAL_ACC: 48.07, eval #45/50 start
2025-07-24 18:59:05: Sobol, failed: 62, best VAL_ACC: 48.07, eval #46/50 start
2025-07-24 18:59:15: Sobol, failed: 62, best VAL_ACC: 48.07, eval #47/50 start
2025-07-24 18:59:25: Sobol, failed: 62, best VAL_ACC: 48.07, eval #48/50 start
2025-07-24 18:59:35: Sobol, failed: 62, best VAL_ACC: 48.07, eval #49/50 start
2025-07-24 18:59:45: Sobol, failed: 62, best VAL_ACC: 48.07, eval #50/50 start
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:56: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 18:59:57: Sobol, failed: 62, best VAL_ACC: 48.07, starting new job
2025-07-24 19:01:25: Sobol, failed: 62, best VAL_ACC: 48.07, unknown 2 = ∑2/50, started new job
2025-07-24 19:01:26: Sobol, failed: 62, best VAL_ACC: 48.07, unknown 2 = ∑2/50, started new job
2025-07-24 19:01:35: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 2/2 = ∑4/50, started new job
2025-07-24 19:01:35: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 2/2 = ∑4/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:39: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:01:41: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 3/1/12 = ∑16/50, started new job
2025-07-24 19:02:17: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 14/2 = ∑16/50, starting new job
2025-07-24 19:02:22: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 14/2 = ∑16/50, starting new job
2025-07-24 19:03:19: Sobol, failed: 62, best VAL_ACC: 48.07, running 16 = ∑16/50, starting new job
2025-07-24 19:03:19: Sobol, failed: 62, best VAL_ACC: 48.07, running 16 = ∑16/50, starting new job
2025-07-24 19:03:36: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, started new job
2025-07-24 19:03:37: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, started new job
2025-07-24 19:03:45: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:46: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:49: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:49: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:49: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:54: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:54: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:54: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:55: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:03:55: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 16/2 = ∑18/50, starting new job
2025-07-24 19:04:16: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 18/2 = ∑20/50, started new job
2025-07-24 19:04:17: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 18/2 = ∑20/50, started new job
2025-07-24 19:05:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 20 = ∑20/50, starting new job
2025-07-24 19:05:25: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 20/1 = ∑21/50, started new job
2025-07-24 19:05:26: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 20/1 = ∑21/50, starting new job
2025-07-24 19:05:30: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 21/1 = ∑22/50, starting new job
2025-07-24 19:05:30: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 21/1 = ∑22/50, started new job
2025-07-24 19:05:32: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 22/3 = ∑25/50, started new job
2025-07-24 19:05:32: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 22/3 = ∑25/50, started new job
2025-07-24 19:05:33: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 22/3 = ∑25/50, started new job
2025-07-24 19:05:36: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/3/1 = ∑26/50, starting new job
2025-07-24 19:05:38: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/3/5 = ∑30/50, started new job
2025-07-24 19:05:38: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/3/5 = ∑30/50, started new job
2025-07-24 19:05:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/7/1 = ∑30/50, started new job
2025-07-24 19:05:40: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/7/1 = ∑30/50, started new job
2025-07-24 19:05:41: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 22/7/1 = ∑30/50, started new job
2025-07-24 19:06:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 30 = ∑30/50, starting new job
2025-07-24 19:06:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 30 = ∑30/50, starting new job
2025-07-24 19:07:08: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 30/1 = ∑31/50, started new job
2025-07-24 19:07:13: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 31/1 = ∑32/50, started new job
2025-07-24 19:07:15: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 31/1 = ∑32/50, starting new job
2025-07-24 19:07:18: Sobol, failed: 62, best , running/pending/unknown 31/1/1 = ∑33/50, started new job
2025-07-24 19:07:33: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 31/3 = ∑34/50, started new job
2025-07-24 19:07:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:43: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:43: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:49: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:49: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, starting new job
2025-07-24 19:07:52: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 34/1 = ∑35/50, started new job
2025-07-24 19:07:58: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 34/2 = ∑36/50, started new job
2025-07-24 19:09:02: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 36/1 = ∑37/50, started new job
2025-07-24 19:09:03: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 36/1 = ∑37/50, starting new job
2025-07-24 19:09:07: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 36/1 = ∑37/50, starting new job
2025-07-24 19:09:15: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 36/1 = ∑37/50, starting new job
2025-07-24 19:09:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 37 = ∑37/50, starting new job
2025-07-24 19:09:42: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 37/2 = ∑39/50, starting new job
2025-07-24 19:09:43: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 37/2 = ∑39/50, started new job
2025-07-24 19:09:44: Sobol, failed: 62, best VAL_ACC: 48.07, running/unknown 37/2 = ∑39/50, started new job
2025-07-24 19:09:48: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 37/2/6 = ∑45/50, started new job
2025-07-24 19:09:49: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 37/2/6 = ∑45/50, started new job
2025-07-24 19:09:49: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending/unknown 37/2/6 = ∑45/50, started new job
2025-07-24 19:09:53: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 37/8 = ∑45/50, started new job
2025-07-24 19:09:53: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 37/8 = ∑45/50, started new job
2025-07-24 19:09:54: Sobol, failed: 62, best VAL_ACC: 48.07, running/pending 37/8 = ∑45/50, started new job
2025-07-24 19:10:43: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed/unknown 44/1/1 = ∑46/50, started new job
2025-07-24 19:10:48: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed/pending/unknown 44/1/1/1 = ∑47/50, started new job
2025-07-24 19:10:55: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed/pending 44/1/3 = ∑48/50, started new job
2025-07-24 19:11:16: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed/unknown 47/1/1 = ∑49/50, started new job
2025-07-24 19:11:35: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed/pending 48/1/1 = ∑50/50, started new job
2025-07-24 19:12:14: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed 49/1 = ∑50/50, new result: 9.88
2025-07-24 19:12:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 49 = ∑49/50, finishing jobs, finished 1 job
2025-07-24 19:13:07: Sobol, failed: 62, best VAL_ACC: 48.07, running 49 = ∑49/50, waiting for 49 jobs
2025-07-24 19:13:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 49 = ∑49/50, new result: 11.35
2025-07-24 19:13:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-24 19:14:09: Sobol, failed: 62, best VAL_ACC: 48.07, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 19:14:27: Sobol, failed: 62, best VAL_ACC: 48.07, running 48 = ∑48/50, waiting for 48 jobs
2025-07-24 19:14:46: Sobol, failed: 62, best VAL_ACC: 48.07, running/completed 47/1 = ∑48/50, new result: 9.83
2025-07-24 19:15:33: Sobol, failed: 62, best VAL_ACC: 48.07, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-24 19:15:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 47 = ∑47/50, waiting for 47 jobs
2025-07-24 19:16:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 47 = ∑47/50, new result: 11.35
2025-07-24 19:16:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-24 19:16:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 46 = ∑46/50, waiting for 46 jobs
2025-07-24 19:17:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 46 = ∑46/50, new result: 0.03
2025-07-24 19:17:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job
2025-07-24 19:17:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:18:11: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:18:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:18:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:19:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:19:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:19:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, waiting for 45 jobs
2025-07-24 19:19:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 45 = ∑45/50, new result: 4.8
2025-07-24 19:20:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job
2025-07-24 19:20:57: Sobol, failed: 62, best VAL_ACC: 48.07, running 44 = ∑44/50, waiting for 44 jobs
2025-07-24 19:21:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 44 = ∑44/50, new result: 9.8
2025-07-24 19:21:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 44 = ∑44/50, new result: 9.72
2025-07-24 19:21:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 44 jobs, finished 2 jobs
2025-07-24 19:22:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:22:28: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:22:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:23:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:23:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:23:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:23:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:24:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:24:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:24:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:25:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:25:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:25:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:26:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:26:26: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:26:45: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:27:03: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:27:22: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:27:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:27:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:28:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:28:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:28:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:29:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:29:31: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:29:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:30:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:30:28: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:30:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:31:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:31:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:31:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:32:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, waiting for 42 jobs
2025-07-24 19:32:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 42 = ∑42/50, new result: 11.55
2025-07-24 19:32:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 42 jobs, finished 1 job
2025-07-24 19:33:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:33:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:33:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:33:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:34:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:34:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:34:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:35:11: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:35:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:35:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:36:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:36:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:36:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:36:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:37:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:37:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:37:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:38:13: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:38:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:38:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:39:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:39:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:39:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:40:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:40:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:40:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:40:54: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:41:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:41:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:41:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:42:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:42:26: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:42:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:43:03: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:43:20: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:43:38: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:43:56: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:44:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:44:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:44:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:45:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, waiting for 41 jobs
2025-07-24 19:45:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 41 = ∑41/50, new result: 10.28
2025-07-24 19:46:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 41 jobs, finished 1 job
2025-07-24 19:46:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:46:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:46:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:47:07: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:47:25: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:47:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:48:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:48:19: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:48:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, waiting for 40 jobs
2025-07-24 19:48:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 40 = ∑40/50, new result: 9.88
2025-07-24 19:49:31: Sobol, failed: 62, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 40 jobs, finished 1 job
2025-07-24 19:49:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 19:50:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 19:50:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 39 = ∑39/50, waiting for 39 jobs
2025-07-24 19:50:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 39 = ∑39/50, new result: 11.35
2025-07-24 19:51:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job
2025-07-24 19:51:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:51:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:51:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:52:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:52:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:52:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:53:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:53:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:53:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:54:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:54:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:54:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:54:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:55:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:55:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, waiting for 38 jobs
2025-07-24 19:55:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 38 = ∑38/50, new result: 18.0
2025-07-24 19:56:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job
2025-07-24 19:56:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 37 = ∑37/50, waiting for 37 jobs
2025-07-24 19:56:56: Sobol, failed: 62, best VAL_ACC: 48.07, running 37 = ∑37/50, new result: 11.35
2025-07-24 19:57:33: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job
2025-07-24 19:57:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:58:02: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:58:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:58:39: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:58:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:59:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:59:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 19:59:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:00:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:00:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:00:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:01:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:01:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:01:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:01:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:02:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:02:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:02:54: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:03:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:03:31: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:03:49: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:04:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:04:27: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:04:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:05:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:05:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:05:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:06:02: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:06:22: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:06:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:07:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:07:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:07:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:07:54: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:08:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:08:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:08:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:09:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:09:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:09:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:10:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:10:19: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:10:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:10:55: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:11:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:11:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:11:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:12:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:12:26: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:12:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:13:03: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:13:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:13:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:13:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:14:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:14:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:14:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:15:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:15:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:15:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:16:07: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:16:33: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:16:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:17:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:17:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:17:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:18:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:18:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:18:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:19:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:19:30: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:19:49: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:20:07: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:20:25: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:20:43: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:21:02: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:21:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:21:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:22:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:22:19: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:22:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:22:56: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:23:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:23:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:23:54: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:24:13: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:24:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:24:51: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:25:13: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:25:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:25:51: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:26:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:26:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:26:49: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:27:07: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:27:26: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:27:45: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:28:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:28:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:28:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:29:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:29:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:29:39: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:29:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:30:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:30:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:30:54: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:31:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:31:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:32:02: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:32:20: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:32:39: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:32:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:33:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:33:35: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:33:53: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:34:12: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:34:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:34:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:35:09: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:35:27: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:35:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:36:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:36:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:36:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:37:00: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:37:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:37:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:37:55: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:38:14: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:38:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:38:51: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:39:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:39:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:39:48: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:40:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:40:25: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:40:44: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:41:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:41:20: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:41:38: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:41:57: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:42:15: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:42:33: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:42:51: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:43:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:43:28: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:43:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:44:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:44:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:44:43: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, waiting for 36 jobs
2025-07-24 20:44:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 36 = ∑36/50, new result: 0.57
2025-07-24 20:45:36: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job
2025-07-24 20:45:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:46:05: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:46:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:46:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:46:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:47:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:47:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:47:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:48:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:48:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:48:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:49:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:49:27: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:49:45: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:50:02: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, waiting for 35 jobs
2025-07-24 20:50:18: Sobol, failed: 62, best VAL_ACC: 48.07, running 35 = ∑35/50, new result: 10.1
2025-07-24 20:50:55: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 35 jobs, finished 1 job
2025-07-24 20:51:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 20:51:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 20:51:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 20:51:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, waiting for 34 jobs
2025-07-24 20:52:16: Sobol, failed: 62, best VAL_ACC: 48.07, running 34 = ∑34/50, new result: 11.35
2025-07-24 20:52:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job
2025-07-24 20:53:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 33 = ∑33/50, waiting for 33 jobs
2025-07-24 20:53:21: Sobol, failed: 62, best VAL_ACC: 48.07, running 33 = ∑33/50, new result: 10.28
2025-07-24 20:53:58: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job
2025-07-24 20:54:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:54:28: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:54:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:55:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:55:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:55:41: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:55:59: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:56:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:56:37: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:56:55: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:57:13: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:57:32: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:57:50: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:58:08: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:58:27: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:58:45: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:59:04: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:59:23: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, waiting for 32 jobs
2025-07-24 20:59:40: Sobol, failed: 62, best VAL_ACC: 48.07, running 32 = ∑32/50, new result: 11.35
2025-07-24 21:00:17: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job
2025-07-24 21:00:28: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:00:46: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:01:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:01:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:01:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:02:01: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:02:20: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:02:39: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:02:57: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:03:15: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:03:34: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:03:52: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:04:10: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:04:29: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:04:47: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:05:06: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:05:24: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:05:42: Sobol, failed: 62, best VAL_ACC: 48.07, running 31 = ∑31/50, waiting for 31 jobs
2025-07-24 21:06:15: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:15: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:16: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:06:17: Sobol, failed: 62, best VAL_ACC: 48.07, timeout/running 12/19 = ∑31/50, job_failed
2025-07-24 21:08:06: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 31 jobs, finished 12 jobs
2025-07-24 21:08:17: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:08:35: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:08:54: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:09:12: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:09:31: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:09:49: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:10:07: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:10:26: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:10:44: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:11:02: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:11:21: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:11:39: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:11:57: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:12:15: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:12:33: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:12:51: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:13:09: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:13:27: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:13:45: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:14:03: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:14:22: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:14:40: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:14:58: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:15:16: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:15:34: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:15:53: Sobol, failed: 74, best VAL_ACC: 48.07, running 19 = ∑19/50, waiting for 19 jobs
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:27: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:28: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:28: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:28: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:16:28: Sobol, failed: 74, best VAL_ACC: 48.07, timeout 19 = ∑19/50, job_failed
2025-07-24 21:19:14: Sobol, failed: 93, best VAL_ACC: 48.07, waiting for 19 jobs, finished 19 jobs
2025-07-24 21:24:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #1/50
2025-07-24 21:25:01: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #2/50
2025-07-24 21:25:24: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #3/50
2025-07-24 21:25:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #4/50
2025-07-24 21:26:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #5/50
2025-07-24 21:26:33: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #6/50
2025-07-24 21:26:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #7/50
2025-07-24 21:27:19: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #8/50
2025-07-24 21:27:42: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #9/50
2025-07-24 21:28:06: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #10/50
2025-07-24 21:28:29: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #11/50
2025-07-24 21:28:52: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #12/50
2025-07-24 21:29:15: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #13/50
2025-07-24 21:29:39: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #14/50
2025-07-24 21:30:03: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #15/50
2025-07-24 21:30:27: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #16/50
2025-07-24 21:30:50: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #17/50
2025-07-24 21:31:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #18/50
2025-07-24 21:31:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #19/50
2025-07-24 21:31:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #20/50
2025-07-24 21:32:22: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #21/50
2025-07-24 21:32:46: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #22/50
2025-07-24 21:33:09: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #23/50
2025-07-24 21:33:33: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #24/50
2025-07-24 21:33:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #25/50
2025-07-24 21:34:20: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #26/50
2025-07-24 21:34:43: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #27/50
2025-07-24 21:35:06: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #28/50
2025-07-24 21:35:30: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #29/50
2025-07-24 21:35:52: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #30/50
2025-07-24 21:36:16: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #31/50
2025-07-24 21:36:40: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #32/50
2025-07-24 21:37:03: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #33/50
2025-07-24 21:37:26: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #34/50
2025-07-24 21:37:49: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #35/50
2025-07-24 21:38:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #36/50
2025-07-24 21:38:38: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #37/50
2025-07-24 21:39:02: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #38/50
2025-07-24 21:39:25: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #39/50
2025-07-24 21:39:50: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #40/50
2025-07-24 21:40:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #41/50
2025-07-24 21:40:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #42/50
2025-07-24 21:41:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #43/50
2025-07-24 21:41:24: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #44/50
2025-07-24 21:41:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #45/50
2025-07-24 21:42:11: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #46/50
2025-07-24 21:42:34: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #47/50
2025-07-24 21:42:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #48/50
2025-07-24 21:43:21: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #49/50
2025-07-24 21:43:44: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, getting new HP set #50/50
2025-07-24 21:44:09: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, requested 50 jobs, got 50, 29.62 s/job
2025-07-24 21:44:20: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #1/50 start
2025-07-24 21:44:33: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #2/50 start
2025-07-24 21:44:46: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #3/50 start
2025-07-24 21:44:58: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #4/50 start
2025-07-24 21:45:11: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #5/50 start
2025-07-24 21:45:24: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #6/50 start
2025-07-24 21:45:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #7/50 start
2025-07-24 21:45:49: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #8/50 start
2025-07-24 21:46:01: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #9/50 start
2025-07-24 21:46:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #10/50 start
2025-07-24 21:46:27: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #11/50 start
2025-07-24 21:46:39: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #12/50 start
2025-07-24 21:46:52: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #13/50 start
2025-07-24 21:47:05: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #14/50 start
2025-07-24 21:47:18: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #15/50 start
2025-07-24 21:47:30: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #16/50 start
2025-07-24 21:47:43: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #17/50 start
2025-07-24 21:47:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #18/50 start
2025-07-24 21:48:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #19/50 start
2025-07-24 21:48:22: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #20/50 start
2025-07-24 21:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #21/50 start
2025-07-24 21:48:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #22/50 start
2025-07-24 21:49:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #23/50 start
2025-07-24 21:49:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #24/50 start
2025-07-24 21:49:27: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #25/50 start
2025-07-24 21:49:40: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #26/50 start
2025-07-24 21:49:53: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #27/50 start
2025-07-24 21:50:05: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #28/50 start
2025-07-24 21:50:18: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #29/50 start
2025-07-24 21:50:31: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #30/50 start
2025-07-24 21:50:44: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #31/50 start
2025-07-24 21:50:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #32/50 start
2025-07-24 21:51:09: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #33/50 start
2025-07-24 21:51:22: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #34/50 start
2025-07-24 21:51:34: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #35/50 start
2025-07-24 21:51:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #36/50 start
2025-07-24 21:51:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #37/50 start
2025-07-24 21:52:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #38/50 start
2025-07-24 21:52:25: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #39/50 start
2025-07-24 21:52:38: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #40/50 start
2025-07-24 21:52:51: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #41/50 start
2025-07-24 21:53:04: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #42/50 start
2025-07-24 21:53:16: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #43/50 start
2025-07-24 21:53:29: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #44/50 start
2025-07-24 21:53:41: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #45/50 start
2025-07-24 21:53:54: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #46/50 start
2025-07-24 21:54:07: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #47/50 start
2025-07-24 21:54:20: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #48/50 start
2025-07-24 21:54:32: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #49/50 start
2025-07-24 21:54:45: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, eval #50/50 start
2025-07-24 21:54:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:54:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:54:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:54:59: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:55:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, starting new job
2025-07-24 21:56:58: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, unknown 1 = ∑1/50, started new job
2025-07-24 21:57:04: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/unknown 1/2 = ∑3/50, started new job
2025-07-24 21:57:04: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/unknown 1/2 = ∑3/50, started new job
2025-07-24 21:57:06: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/1/4 = ∑6/50, started new job
2025-07-24 21:57:08: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/7 = ∑11/50, started new job
2025-07-24 21:57:08: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/7 = ∑11/50, started new job
2025-07-24 21:57:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/11 = ∑15/50, started new job
2025-07-24 21:57:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/11 = ∑15/50, started new job
2025-07-24 21:57:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/12 = ∑16/50, started new job
2025-07-24 21:57:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/11 = ∑15/50, started new job
2025-07-24 21:57:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/11 = ∑15/50, started new job
2025-07-24 21:57:11: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/12 = ∑16/50, started new job
2025-07-24 21:57:11: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/12 = ∑16/50, started new job
2025-07-24 21:57:12: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/12 = ∑16/50, started new job
2025-07-24 21:57:12: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/3/12 = ∑16/50, started new job
2025-07-24 21:57:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/pending/unknown 1/11/4 = ∑16/50, started new job
2025-07-24 21:58:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running 16 = ∑16/50, starting new job
2025-07-24 21:59:32: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 15/1/1 = ∑17/50, started new job
2025-07-24 21:59:34: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 15/1/1 = ∑17/50, starting new job
2025-07-24 21:59:34: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 15/1/1 = ∑17/50, starting new job
2025-07-24 21:59:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 15/1/1 = ∑17/50, starting new job
2025-07-24 21:59:45: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:53: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:56: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 21:59:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 22:01:38: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 16/1 = ∑17/50, starting new job
2025-07-24 22:01:42: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 16/1/3 = ∑20/50, started new job
2025-07-24 22:01:43: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 19/1 = ∑20/50, started new job
2025-07-24 22:01:43: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 19/1 = ∑20/50, started new job
2025-07-24 22:01:44: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed 19/1 = ∑20/50, starting new job
2025-07-24 22:01:52: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 19/1/1 = ∑21/50, started new job
2025-07-24 22:02:00: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending/unknown 19/1/1/1 = ∑22/50, started new job
2025-07-24 22:02:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending/unknown 19/1/2/9 = ∑31/50, started new job
2025-07-24 22:02:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending/unknown 19/1/2/9 = ∑31/50, started new job
2025-07-24 22:02:13: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending/unknown 19/1/2/9 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 21/1/9 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 21/1/9 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 21/1/9 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/unknown 21/1/9 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 29/1/1 = ∑31/50, started new job
2025-07-24 22:02:14: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, running/completed/pending 29/1/1 = ∑31/50, started new job
2025-07-24 22:03:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 6/25/2 = ∑33/50, started new job
2025-07-24 22:03:50: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 6/27 = ∑33/50, started new job
2025-07-24 22:04:57: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:07: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:08: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:08: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:09: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:09: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:05:10: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 9/24 = ∑33/50, starting new job
2025-07-24 22:06:23: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:06:23: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:06:25: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:06:27: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:06:32: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:07:17: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 13/20 = ∑33/50, starting new job
2025-07-24 22:07:21: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending 16/17/1 = ∑34/50, started new job
2025-07-24 22:07:28: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 17/17/2 = ∑36/50, started new job
2025-07-24 22:07:29: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 17/17/2 = ∑36/50, started new job
2025-07-24 22:07:30: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 17/17/7 = ∑41/50, started new job
2025-07-24 22:07:31: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 17/17/7 = ∑41/50, started new job
2025-07-24 22:07:31: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending 17/17/7 = ∑41/50, started new job
2025-07-24 22:07:32: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending 17/17/7 = ∑41/50, started new job
2025-07-24 22:07:32: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending 17/17/7 = ∑41/50, started new job
2025-07-24 22:08:25: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 17/24 = ∑41/50, starting new job
2025-07-24 22:08:28: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 17/24 = ∑41/50, starting new job
2025-07-24 22:08:41: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending/unknown 18/23/1/1 = ∑43/50, started new job
2025-07-24 22:08:42: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending/unknown 18/23/1/2 = ∑44/50, started new job
2025-07-24 22:08:43: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending/unknown 18/23/1/2 = ∑44/50, started new job
2025-07-24 22:08:46: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending/unknown 18/23/3/2 = ∑46/50, started new job
2025-07-24 22:08:49: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending/unknown 18/23/3/2 = ∑46/50, started new job
2025-07-24 22:09:47: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 19/27/1 = ∑47/50, started new job
2025-07-24 22:10:19: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 19/28 = ∑47/50, starting new job
2025-07-24 22:10:39: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 20/27/2 = ∑49/50, started new job
2025-07-24 22:10:40: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/pending 20/27/2 = ∑49/50, started new job
2025-07-24 22:11:27: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running/unknown 21/28/1 = ∑50/50, started new job
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 89.5
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 81.86
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 57.54
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 82.29
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 65.85
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 75.34
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 65.44
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 90.18
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 57.47
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 20.22
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 56.53
2025-07-24 22:12:35: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 73.65
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 63.75
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 73.26
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 73.02
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 82.38
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 28.8
2025-07-24 22:12:36: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 57.66
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 66.07
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 80.02
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 73.24
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 56.7
2025-07-24 22:12:37: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 80.22
2025-07-24 22:12:38: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 81.4
2025-07-24 22:12:41: BoTorchGenerator, failed: 93, best VAL_ACC: 48.07, completed/running 25/25 = ∑50/50, new result: 74.1
2025-07-24 22:25:12: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, finishing jobs, finished 25 jobs
2025-07-24 22:25:32: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 65.41
2025-07-24 22:25:32: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 66.2
2025-07-24 22:25:32: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 57.82
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 40.28
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 79.62
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 57.6
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 81.05
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 74.13
2025-07-24 22:25:33: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 58.04
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 43.02
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 81.83
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 81.12
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 91.54
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 75.12
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 36.1
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 58.83
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 71.96
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 95.32
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 72.13
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 62.96
2025-07-24 22:25:34: BoTorchGenerator, failed: 93, best VAL_ACC: 90.18, running/completed 10/15 = ∑25/50, new result: 71.81
2025-07-24 22:34:15: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, finishing previous jobs (25), finished 21 jobs
2025-07-24 22:34:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, waiting for 4 jobs
2025-07-24 22:34:50: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, new result: 30.49
2025-07-24 22:34:51: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, new result: 58.25
2025-07-24 22:34:51: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, new result: 57.98
2025-07-24 22:34:52: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, completed 4 = ∑4/50, new result: 58.05
2025-07-24 22:36:36: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, waiting for 4 jobs, finished 4 jobs
2025-07-24 22:43:54: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #1/50
2025-07-24 22:44:24: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #2/50
2025-07-24 22:44:52: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #3/50
2025-07-24 22:45:21: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #4/50
2025-07-24 22:45:50: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #5/50
2025-07-24 22:46:19: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #6/50
2025-07-24 22:46:48: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #7/50
2025-07-24 22:47:16: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #8/50
2025-07-24 22:47:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #9/50
2025-07-24 22:48:14: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #10/50
2025-07-24 22:48:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #11/50
2025-07-24 22:49:12: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #12/50
2025-07-24 22:49:41: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #13/50
2025-07-24 22:50:10: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #14/50
2025-07-24 22:50:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #15/50
2025-07-24 22:51:07: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #16/50
2025-07-24 22:51:36: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #17/50
2025-07-24 22:52:05: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #18/50
2025-07-24 22:52:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #19/50
2025-07-24 22:53:04: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #20/50
2025-07-24 22:53:32: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #21/50
2025-07-24 22:54:02: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #22/50
2025-07-24 22:54:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #23/50
2025-07-24 22:55:01: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #24/50
2025-07-24 22:55:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #25/50
2025-07-24 22:56:00: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #26/50
2025-07-24 22:56:29: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #27/50
2025-07-24 22:56:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #28/50
2025-07-24 22:57:26: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #29/50
2025-07-24 22:57:55: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #30/50
2025-07-24 22:58:25: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #31/50
2025-07-24 22:58:54: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #32/50
2025-07-24 22:59:24: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #33/50
2025-07-24 22:59:53: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #34/50
2025-07-24 23:00:22: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #35/50
2025-07-24 23:00:50: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #36/50
2025-07-24 23:01:19: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #37/50
2025-07-24 23:01:47: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #38/50
2025-07-24 23:02:17: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #39/50
2025-07-24 23:02:46: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #40/50
2025-07-24 23:03:16: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #41/50
2025-07-24 23:03:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #42/50
2025-07-24 23:04:14: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #43/50
2025-07-24 23:04:44: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #44/50
2025-07-24 23:05:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #45/50
2025-07-24 23:05:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #46/50
2025-07-24 23:06:10: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #47/50
2025-07-24 23:06:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #48/50
2025-07-24 23:07:08: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #49/50
2025-07-24 23:07:21: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #49/50
2025-07-24 23:08:21: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #49/50
2025-07-24 23:08:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #49/50
2025-07-24 23:09:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #49/50
2025-07-24 23:10:08: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #50/50
2025-07-24 23:10:46: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #50/50
2025-07-24 23:11:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, getting new HP set #50/50
2025-07-24 23:12:06: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, requested 50 jobs, got 50, 42.18 s/job
2025-07-24 23:12:19: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #1/50 start
2025-07-24 23:12:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #2/50 start
2025-07-24 23:12:50: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #3/50 start
2025-07-24 23:13:06: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #4/50 start
2025-07-24 23:13:22: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #5/50 start
2025-07-24 23:13:38: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #6/50 start
2025-07-24 23:13:54: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #7/50 start
2025-07-24 23:14:09: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #8/50 start
2025-07-24 23:14:26: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #9/50 start
2025-07-24 23:14:41: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #10/50 start
2025-07-24 23:14:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #11/50 start
2025-07-24 23:15:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #12/50 start
2025-07-24 23:15:29: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #13/50 start
2025-07-24 23:15:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #14/50 start
2025-07-24 23:16:01: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #15/50 start
2025-07-24 23:16:17: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #16/50 start
2025-07-24 23:16:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #17/50 start
2025-07-24 23:16:49: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #18/50 start
2025-07-24 23:17:05: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #19/50 start
2025-07-24 23:17:21: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #20/50 start
2025-07-24 23:17:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #21/50 start
2025-07-24 23:17:53: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #22/50 start
2025-07-24 23:18:08: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #23/50 start
2025-07-24 23:18:24: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #24/50 start
2025-07-24 23:18:40: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #25/50 start
2025-07-24 23:18:56: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #26/50 start
2025-07-24 23:19:11: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #27/50 start
2025-07-24 23:19:27: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #28/50 start
2025-07-24 23:19:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #29/50 start
2025-07-24 23:19:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #30/50 start
2025-07-24 23:20:14: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #31/50 start
2025-07-24 23:20:28: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #32/50 start
2025-07-24 23:20:44: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #33/50 start
2025-07-24 23:20:59: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #34/50 start
2025-07-24 23:21:14: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #35/50 start
2025-07-24 23:21:29: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #36/50 start
2025-07-24 23:21:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #37/50 start
2025-07-24 23:22:00: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #38/50 start
2025-07-24 23:22:15: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #39/50 start
2025-07-24 23:22:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #40/50 start
2025-07-24 23:22:47: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #41/50 start
2025-07-24 23:23:04: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #42/50 start
2025-07-24 23:23:19: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #43/50 start
2025-07-24 23:23:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #44/50 start
2025-07-24 23:23:52: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #45/50 start
2025-07-24 23:24:09: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #46/50 start
2025-07-24 23:24:25: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #47/50 start
2025-07-24 23:24:41: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #48/50 start
2025-07-24 23:24:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #49/50 start
2025-07-24 23:25:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, eval #50/50 start
2025-07-24 23:25:30: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:31: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:25:32: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, starting new job
2025-07-24 23:28:02: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, unknown 2 = ∑2/50, started new job
2025-07-24 23:28:02: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, unknown 2 = ∑2/50, started new job
2025-07-24 23:28:05: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 1/2 = ∑3/50, started new job
2025-07-24 23:28:07: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending 6 = ∑6/50, started new job
2025-07-24 23:28:08: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/1 = ∑7/50, started new job
2025-07-24 23:28:08: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/4 = ∑10/50, started new job
2025-07-24 23:28:09: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/7 = ∑13/50, started new job
2025-07-24 23:28:10: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/10 = ∑16/50, started new job
2025-07-24 23:28:10: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/10 = ∑16/50, started new job
2025-07-24 23:28:11: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/10 = ∑16/50, started new job
2025-07-24 23:28:11: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/10 = ∑16/50, started new job
2025-07-24 23:28:12: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 6/10 = ∑16/50, started new job
2025-07-24 23:28:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 13/3 = ∑16/50, started new job
2025-07-24 23:28:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 13/3 = ∑16/50, started new job
2025-07-24 23:28:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 13/3 = ∑16/50, started new job
2025-07-24 23:28:13: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, pending/unknown 13/3 = ∑16/50, started new job
2025-07-24 23:31:23: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:24: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:26: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:26: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:30: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:40: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:43: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:46: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:54: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:31:59: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:32:01: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:32:01: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:32:01: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 8/8 = ∑16/50, starting new job
2025-07-24 23:33:58: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 8/8/2 = ∑18/50, started new job
2025-07-24 23:33:58: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 8/8/2 = ∑18/50, started new job
2025-07-24 23:34:02: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 8/8/2/2 = ∑20/50, started new job
2025-07-24 23:34:03: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 12/8 = ∑20/50, started new job
2025-07-24 23:34:11: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 12/8/1 = ∑21/50, started new job
2025-07-24 23:34:20: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 12/8/2 = ∑22/50, started new job
2025-07-24 23:34:23: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 12/8/2/1 = ∑23/50, started new job
2025-07-24 23:34:27: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 12/8/3/2 = ∑25/50, started new job
2025-07-24 23:34:28: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 12/8/3/2 = ∑25/50, started new job
2025-07-24 23:34:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 18/8 = ∑26/50, started new job
2025-07-24 23:34:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/8/1 = ∑27/50, started new job
2025-07-24 23:34:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/8/1 = ∑27/50, starting new job
2025-07-24 23:34:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/8/2 = ∑28/50, started new job
2025-07-24 23:34:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/8/2 = ∑28/50, starting new job
2025-07-24 23:34:41: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 18/8/3/1 = ∑30/50, started new job
2025-07-24 23:34:43: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 18/8/3/1 = ∑30/50, started new job
2025-07-24 23:37:30: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 16/14/2 = ∑32/50, started new job
2025-07-24 23:37:30: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 16/14/2 = ∑32/50, started new job
2025-07-24 23:37:41: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:43: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:43: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:48: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:55: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:37:58: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 15/15/2 = ∑32/50, starting new job
2025-07-24 23:38:04: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 16/16 = ∑32/50, starting new job
2025-07-24 23:38:06: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 16/16 = ∑32/50, starting new job
2025-07-24 23:38:07: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 16/16 = ∑32/50, starting new job
2025-07-24 23:38:07: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 16/16 = ∑32/50, starting new job
2025-07-24 23:40:26: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 13/19 = ∑32/50, starting new job
2025-07-24 23:40:36: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 13/19/4 = ∑36/50, started new job
2025-07-24 23:40:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 13/19/5 = ∑37/50, started new job
2025-07-24 23:40:38: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 13/19/5 = ∑37/50, started new job
2025-07-24 23:40:38: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 13/19/5 = ∑37/50, started new job
2025-07-24 23:40:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 13/19/5 = ∑37/50, started new job
2025-07-24 23:40:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 13/19/5/1 = ∑38/50, started new job
2025-07-24 23:40:49: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 13/19/6/2 = ∑40/50, started new job
2025-07-24 23:40:49: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 13/19/6/2 = ∑40/50, started new job
2025-07-24 23:40:52: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 13/19/6/2 = ∑40/50, starting new job
2025-07-24 23:40:55: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 21/19/2 = ∑42/50, started new job
2025-07-24 23:40:58: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 21/19/1/1 = ∑42/50, started new job
2025-07-24 23:40:59: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending/unknown 21/19/1/3 = ∑44/50, started new job
2025-07-24 23:41:03: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 21/19/4 = ∑44/50, started new job
2025-07-24 23:43:19: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 21/23/1 = ∑45/50, starting new job
2025-07-24 23:43:20: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 21/23/1 = ∑45/50, started new job
2025-07-24 23:43:20: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/pending 21/23/1 = ∑45/50, starting new job
2025-07-24 23:43:45: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 21/24/1 = ∑46/50, started new job
2025-07-24 23:44:39: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 21/25 = ∑46/50, starting new job
2025-07-24 23:44:42: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 21/25 = ∑46/50, starting new job
2025-07-24 23:45:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/28/1 = ∑47/50, started new job
2025-07-24 23:45:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 18/28/2 = ∑48/50, started new job
2025-07-24 23:46:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 19/29/2 = ∑50/50, started new job
2025-07-24 23:46:57: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed/unknown 19/29/2 = ∑50/50, started new job
2025-07-24 23:48:32: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 22.74
2025-07-24 23:48:32: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 64.59
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 74.78
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 39.31
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 20.74
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 28.38
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 61.21
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 69.97
2025-07-24 23:48:33: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 56.17
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 90.3
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 92.12
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 88.85
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 11.39
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 47.28
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 11.35
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 82.59
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 77.95
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 93.2
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 27.32
2025-07-24 23:48:32: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 92.86
2025-07-24 23:48:34: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 83.92
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 19.75
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 39.05
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 42.84
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 76.35
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 92.24
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 98.08
2025-07-24 23:48:35: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 79.15
2025-07-24 23:48:36: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 52.58
2025-07-24 23:48:36: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 90.71
2025-07-24 23:48:37: BoTorchGenerator, failed: 93, best VAL_ACC: 95.32, running/completed 19/31 = ∑50/50, new result: 94.47
2025-07-25 00:15:56: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 11/8 = ∑19/50, finishing jobs, finished 31 jobs
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 98.29
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 91.3
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 85.34
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 98.08
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 98.22
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 90.66
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 98.19
2025-07-25 00:16:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 86.85
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 91.06
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 92.73
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 85.74
2025-07-25 00:16:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.08, completed/running 12/7 = ∑19/50, new result: 93.15
2025-07-25 00:21:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, finishing previous jobs (19), finished 12 jobs
2025-07-25 00:21:58: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:22:23: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:22:48: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:23:13: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:23:38: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:24:03: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:24:29: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:24:54: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:25:19: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:25:45: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:26:10: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:26:34: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:27:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:27:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:27:50: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:28:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:28:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:29:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:29:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:29:55: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:30:20: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:30:45: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:31:10: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:31:35: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:32:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:32:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:32:50: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:33:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:33:40: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:34:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:34:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:34:55: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:35:20: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:35:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:36:11: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:36:36: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:37:02: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:37:28: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:37:53: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:38:17: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:38:43: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:39:09: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:39:34: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:40:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:40:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:40:50: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:41:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:41:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:42:06: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:42:32: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:42:58: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:43:23: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:43:48: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:44:14: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:44:44: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:45:09: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:45:52: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:46:38: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:47:38: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:48:18: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:48:47: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:49:14: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:49:39: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:50:07: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:50:34: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:51:07: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:51:34: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:52:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:52:28: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:52:57: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:53:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:53:58: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:54:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:54:51: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:55:18: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:55:45: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:56:11: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:56:38: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:57:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:57:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:57:56: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:58:21: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:58:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:59:11: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 00:59:37: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:00:04: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:00:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:00:55: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:01:20: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:01:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:02:12: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:02:44: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:03:10: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:03:35: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:04:01: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:04:27: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:04:52: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:05:18: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:05:45: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:06:11: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, waiting for 7 jobs
2025-07-25 01:06:32: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 7 = ∑7/50, new result: 9.58
2025-07-25 01:07:29: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 7 jobs, finished 1 job
2025-07-25 01:07:45: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:08:10: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:08:40: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:09:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:09:31: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:09:58: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:10:23: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:10:49: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:11:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:11:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:12:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:12:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:13:07: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:13:34: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:13:59: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:14:24: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:14:49: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:15:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:15:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:16:07: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:16:32: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:16:57: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:17:23: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:17:51: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:18:16: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:18:43: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:19:08: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:19:33: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:20:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:20:26: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:20:51: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:21:17: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:21:41: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:22:06: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:22:31: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:22:56: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:23:21: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:23:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:24:12: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:24:37: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:25:02: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:25:28: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:25:53: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:26:19: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:26:44: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:27:09: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:27:35: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:28:00: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:28:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:28:49: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:29:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:29:40: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:30:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:30:31: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:30:58: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:31:25: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:31:50: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:32:15: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:32:40: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:33:05: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:33:30: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:33:55: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:34:20: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:34:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:35:12: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:35:38: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:36:03: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:36:29: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:36:54: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:37:19: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:37:46: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, running 6 = ∑6/50, waiting for 6 jobs
2025-07-25 01:38:23: BoTorchGenerator, failed: 93, best VAL_ACC: 98.29, timeout/running 1/5 = ∑6/50, job_failed
2025-07-25 01:38:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 6 jobs, finished 1 job
2025-07-25 01:39:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:39:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:40:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:40:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:40:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:41:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:42:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:42:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:43:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:43:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:43:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:44:23: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:44:48: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:45:13: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:45:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:46:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:46:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:46:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:47:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:47:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 5 = ∑5/50, waiting for 5 jobs
2025-07-25 01:59:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #1/50
2025-07-25 02:00:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #2/50
2025-07-25 02:01:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #3/50
2025-07-25 02:01:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #4/50
2025-07-25 02:02:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #5/50
2025-07-25 02:02:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #6/50
2025-07-25 02:03:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #7/50
2025-07-25 02:04:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #8/50
2025-07-25 02:04:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #9/50
2025-07-25 02:05:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #10/50
2025-07-25 02:05:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #11/50
2025-07-25 02:06:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #12/50
2025-07-25 02:06:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #13/50
2025-07-25 02:07:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #14/50
2025-07-25 02:08:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #15/50
2025-07-25 02:08:43: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #16/50
2025-07-25 02:09:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #17/50
2025-07-25 02:09:55: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #18/50
2025-07-25 02:10:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #19/50
2025-07-25 02:11:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #20/50
2025-07-25 02:11:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #21/50
2025-07-25 02:12:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #22/50
2025-07-25 02:12:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #23/50
2025-07-25 02:13:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #24/50
2025-07-25 02:14:01: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #25/50
2025-07-25 02:14:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #26/50
2025-07-25 02:15:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #27/50
2025-07-25 02:15:45: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #28/50
2025-07-25 02:16:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #29/50
2025-07-25 02:16:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #30/50
2025-07-25 02:17:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #31/50
2025-07-25 02:18:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #32/50
2025-07-25 02:18:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #33/50
2025-07-25 02:19:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #34/50
2025-07-25 02:19:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #35/50
2025-07-25 02:20:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #36/50
2025-07-25 02:21:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #37/50
2025-07-25 02:21:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #38/50
2025-07-25 02:22:12: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #39/50
2025-07-25 02:22:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #40/50
2025-07-25 02:23:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #41/50
2025-07-25 02:23:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #42/50
2025-07-25 02:24:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #43/50
2025-07-25 02:25:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #44/50
2025-07-25 02:25:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #45/50
2025-07-25 02:26:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #46/50
2025-07-25 02:26:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #47/50
2025-07-25 02:27:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #48/50
2025-07-25 02:28:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #49/50
2025-07-25 02:28:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, getting new HP set #50/50
2025-07-25 02:29:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, requested 50 jobs, got 50, 49.21 s/job
2025-07-25 02:29:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #1/50 start
2025-07-25 02:29:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #2/50 start
2025-07-25 02:30:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #3/50 start
2025-07-25 02:30:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #4/50 start
2025-07-25 02:30:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #5/50 start
2025-07-25 02:31:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #6/50 start
2025-07-25 02:31:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #7/50 start
2025-07-25 02:31:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #8/50 start
2025-07-25 02:32:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #9/50 start
2025-07-25 02:32:28: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #10/50 start
2025-07-25 02:32:48: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #11/50 start
2025-07-25 02:33:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #12/50 start
2025-07-25 02:33:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #13/50 start
2025-07-25 02:33:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #14/50 start
2025-07-25 02:34:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #15/50 start
2025-07-25 02:34:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #16/50 start
2025-07-25 02:34:43: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #17/50 start
2025-07-25 02:35:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #18/50 start
2025-07-25 02:35:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #19/50 start
2025-07-25 02:35:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #20/50 start
2025-07-25 02:36:01: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #21/50 start
2025-07-25 02:36:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #22/50 start
2025-07-25 02:36:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #23/50 start
2025-07-25 02:37:01: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #24/50 start
2025-07-25 02:37:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #25/50 start
2025-07-25 02:37:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #26/50 start
2025-07-25 02:37:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #27/50 start
2025-07-25 02:38:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #28/50 start
2025-07-25 02:38:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #29/50 start
2025-07-25 02:38:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #30/50 start
2025-07-25 02:39:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #31/50 start
2025-07-25 02:39:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #32/50 start
2025-07-25 02:39:55: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #33/50 start
2025-07-25 02:40:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #34/50 start
2025-07-25 02:40:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #35/50 start
2025-07-25 02:40:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #36/50 start
2025-07-25 02:41:12: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #37/50 start
2025-07-25 02:41:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #38/50 start
2025-07-25 02:41:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #39/50 start
2025-07-25 02:42:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #40/50 start
2025-07-25 02:42:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #41/50 start
2025-07-25 02:42:48: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #42/50 start
2025-07-25 02:43:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #43/50 start
2025-07-25 02:43:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #44/50 start
2025-07-25 02:43:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #45/50 start
2025-07-25 02:44:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #46/50 start
2025-07-25 02:44:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #47/50 start
2025-07-25 02:44:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #48/50 start
2025-07-25 02:45:12: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #49/50 start
2025-07-25 02:45:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, eval #50/50 start
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:45:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, starting new job
2025-07-25 02:48:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, unknown 1 = ∑1/50, started new job
2025-07-25 02:48:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, pending/unknown 1/1 = ∑2/50, started new job
2025-07-25 02:49:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/unknown 2/2 = ∑4/50, started new job
2025-07-25 02:49:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/1/2 = ∑5/50, started new job
2025-07-25 02:49:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/1/6 = ∑9/50, started new job
2025-07-25 02:49:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/1/11 = ∑14/50, started new job
2025-07-25 02:49:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/1/11 = ∑14/50, started new job
2025-07-25 02:49:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/1/12 = ∑15/50, started new job
2025-07-25 02:49:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 2/13 = ∑15/50, started new job
2025-07-25 02:49:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 2/13 = ∑15/50, started new job
2025-07-25 02:49:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 2/13 = ∑15/50, started new job
2025-07-25 02:49:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/13/1 = ∑16/50, started new job
2025-07-25 02:49:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 2/13 = ∑15/50, started new job
2025-07-25 02:49:36: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/13/1 = ∑16/50, started new job
2025-07-25 02:49:36: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/13/1 = ∑16/50, started new job
2025-07-25 02:49:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 2/13/1 = ∑16/50, started new job
2025-07-25 02:50:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 16 = ∑16/50, starting new job
2025-07-25 02:50:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 16 = ∑16/50, starting new job
2025-07-25 02:52:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 16 = ∑16/50, starting new job
2025-07-25 02:52:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 16 = ∑16/50, starting new job
2025-07-25 02:52:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 16 = ∑16/50, starting new job
2025-07-25 02:53:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, started new job
2025-07-25 02:53:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, started new job
2025-07-25 02:54:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:17: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:54:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 18 = ∑18/50, starting new job
2025-07-25 02:55:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/unknown 18/2 = ∑20/50, started new job
2025-07-25 02:55:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/unknown 18/2 = ∑20/50, started new job
2025-07-25 02:55:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 18/3 = ∑21/50, started new job
2025-07-25 02:58:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 21 = ∑21/50, starting new job
2025-07-25 02:58:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 21 = ∑21/50, starting new job
2025-07-25 02:58:13: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 23 = ∑23/50, started new job
2025-07-25 02:58:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 23 = ∑23/50, started new job
2025-07-25 02:58:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 23/1 = ∑24/50, started new job
2025-07-25 02:58:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 23/1/3 = ∑27/50, started new job
2025-07-25 02:58:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 23/1/3 = ∑27/50, started new job
2025-07-25 02:58:28: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 23/1/3 = ∑27/50, started new job
2025-07-25 02:58:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 23/4/3 = ∑30/50, started new job
2025-07-25 02:58:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 23/7 = ∑30/50, started new job
2025-07-25 02:58:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 23/7 = ∑30/50, started new job
2025-07-25 02:58:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending/unknown 23/7/2 = ∑32/50, started new job
2025-07-25 02:58:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/pending 23/9 = ∑32/50, started new job
2025-07-25 03:00:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/1 = ∑32/50, starting new job
2025-07-25 03:00:23: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/1 = ∑32/50, starting new job
2025-07-25 03:01:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 30/2/1 = ∑33/50, started new job
2025-07-25 03:01:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 30/2/2 = ∑34/50, started new job
2025-07-25 03:02:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/3 = ∑34/50, starting new job
2025-07-25 03:02:55: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/3 = ∑34/50, starting new job
2025-07-25 03:02:55: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/3 = ∑34/50, starting new job
2025-07-25 03:02:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/3 = ∑34/50, starting new job
2025-07-25 03:02:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 31/3 = ∑34/50, starting new job
2025-07-25 03:03:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 31/3/2 = ∑36/50, starting new job
2025-07-25 03:03:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 31/3/2 = ∑36/50, starting new job
2025-07-25 03:03:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, started new job
2025-07-25 03:03:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, starting new job
2025-07-25 03:03:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, started new job
2025-07-25 03:03:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, starting new job
2025-07-25 03:03:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, starting new job
2025-07-25 03:03:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 31/3/2 = ∑36/50, starting new job
2025-07-25 03:06:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 29/7 = ∑36/50, starting new job
2025-07-25 03:06:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 29/7/2 = ∑38/50, started new job
2025-07-25 03:06:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 29/7/3 = ∑39/50, started new job
2025-07-25 03:06:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 29/7/4 = ∑40/50, started new job
2025-07-25 03:06:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending/unknown 29/7/4/1 = ∑41/50, started new job
2025-07-25 03:06:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending/unknown 29/7/4/1 = ∑41/50, started new job
2025-07-25 03:06:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending/unknown 29/7/4/2 = ∑42/50, started new job
2025-07-25 03:06:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 29/7/7 = ∑43/50, started new job
2025-07-25 03:06:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 29/7/10 = ∑46/50, started new job
2025-07-25 03:06:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 29/7/10 = ∑46/50, started new job
2025-07-25 03:06:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 29/7/10 = ∑46/50, started new job
2025-07-25 03:06:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending 39/7/1 = ∑47/50, started new job
2025-07-25 03:08:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/7 = ∑47/50, starting new job
2025-07-25 03:08:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/7 = ∑47/50, starting new job
2025-07-25 03:09:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 39/8/1 = ∑48/50, started new job
2025-07-25 03:10:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/unknown 39/9/1 = ∑49/50, started new job
2025-07-25 03:10:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed/pending/unknown 39/9/1/1 = ∑50/50, started new job
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 95.25
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 97.44
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 90.64
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 10.32
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 97.66
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 91.74
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 96.85
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 93.78
2025-07-25 03:13:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 97.15
2025-07-25 03:13:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 40/10 = ∑50/50, new result: 10.32
2025-07-25 03:19:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 40 = ∑40/50, finishing jobs, finished 10 jobs
2025-07-25 03:19:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 40 = ∑40/50, new result: 64.67
2025-07-25 03:20:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 39 = ∑39/50, finishing previous jobs (40), finished 1 job
2025-07-25 03:20:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 39 = ∑39/50, waiting for 39 jobs
2025-07-25 03:21:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 39 = ∑39/50, new result: 10.32
2025-07-25 03:22:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job
2025-07-25 03:22:36: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:23:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:23:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:23:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:24:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:24:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:25:20: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:25:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:26:17: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:26:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:27:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:27:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:28:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:28:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:29:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:29:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:29:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:30:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:30:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:31:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:31:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:32:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:32:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:33:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 03:33:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 38 = ∑38/50, new result: 97.93
2025-07-25 03:34:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job
2025-07-25 03:35:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 37 = ∑37/50, waiting for 37 jobs
2025-07-25 03:35:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 37 = ∑37/50, waiting for 37 jobs
2025-07-25 03:36:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 36/1 = ∑37/50, new result: 98.09
2025-07-25 03:37:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job
2025-07-25 03:37:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:37:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:38:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:38:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:39:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:39:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:40:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:40:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:41:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:41:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:42:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 03:42:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 36 = ∑36/50, new result: 95.32
2025-07-25 03:43:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job
2025-07-25 03:43:50: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, waiting for 35 jobs
2025-07-25 03:44:17: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, waiting for 35 jobs
2025-07-25 03:44:45: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, waiting for 35 jobs
2025-07-25 03:45:12: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, waiting for 35 jobs
2025-07-25 03:45:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 35 = ∑35/50, new result: 87.92
2025-07-25 03:46:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 35 jobs, finished 1 job
2025-07-25 03:46:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:47:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:47:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:48:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:48:45: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:49:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:49:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:50:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:50:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:51:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:51:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:52:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:52:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:52:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:53:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:53:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:54:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:54:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:55:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:55:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:56:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:56:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:57:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:57:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:58:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:58:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:59:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 03:59:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:00:00: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:00:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:00:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:01:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:01:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:02:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:02:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:03:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:03:41: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:04:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:04:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:05:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:05:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:06:00: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:06:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:06:54: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, waiting for 34 jobs
2025-07-25 04:07:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 34 = ∑34/50, new result: 97.0
2025-07-25 04:08:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job
2025-07-25 04:08:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 33 = ∑33/50, waiting for 33 jobs
2025-07-25 04:09:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 33 = ∑33/50, new result: 10.1
2025-07-25 04:10:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job
2025-07-25 04:10:43: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:11:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:11:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:12:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:12:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:13:01: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:13:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:13:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:14:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:14:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:15:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:15:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:16:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:16:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:17:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:17:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:18:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 04:18:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 32 = ∑32/50, new result: 98.19
2025-07-25 04:19:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job
2025-07-25 04:19:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:20:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:20:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:21:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:21:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:22:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:22:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:22:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:23:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 04:23:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 31 = ∑31/50, new result: 97.15
2025-07-25 04:24:51: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job
2025-07-25 04:25:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:25:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:26:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:26:31: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:26:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:27:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:27:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:28:21: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:28:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:29:17: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:29:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:30:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:30:43: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:31:12: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:31:40: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:32:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:32:36: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:33:05: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:33:33: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 04:33:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 30 = ∑30/50, new result: 60.91
2025-07-25 04:35:00: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 30 jobs, finished 1 job
2025-07-25 04:35:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:35:46: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:36:16: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:36:44: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:37:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:37:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:38:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:38:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:39:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 04:39:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 29 = ∑29/50, new result: 9.58
2025-07-25 04:40:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 29 jobs, finished 1 job
2025-07-25 04:40:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:41:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:41:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:42:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:42:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:43:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:43:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:44:03: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:44:30: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:45:01: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:45:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:45:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:46:26: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:46:56: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:47:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 04:47:48: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running/completed 27/1 = ∑28/50, new result: 10.09
2025-07-25 04:48:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 28 jobs, finished 1 job
2025-07-25 04:49:11: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:49:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:50:07: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:50:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:51:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:51:39: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:52:08: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:52:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:53:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:53:35: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:54:04: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:54:32: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:54:59: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:55:27: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:55:57: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:56:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:56:55: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:57:24: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 04:57:52: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 04:58:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 04:58:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 04:59:15: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 04:59:43: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:00:10: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:00:38: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:01:06: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:01:34: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:02:02: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:02:29: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:02:58: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:03:25: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:03:53: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:04:22: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:04:49: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:05:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:05:47: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:06:14: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:06:42: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:07:09: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:07:37: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, running 19 = ∑19/50, waiting for 19 jobs
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:18: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:08:19: BoTorchGenerator, failed: 94, best VAL_ACC: 98.29, timeout/running 18/1 = ∑19/50, job_failed
2025-07-25 05:12:48: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 19 jobs, finished 18 jobs
2025-07-25 05:13:05: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:13:32: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:13:59: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:14:27: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:14:55: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:15:23: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:15:51: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:16:18: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:16:45: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:17:13: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:17:40: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, running 1 = ∑1/50, waiting for 1 job
2025-07-25 05:18:18: BoTorchGenerator, failed: 112, best VAL_ACC: 98.29, timeout 1 = ∑1/50, job_failed
2025-07-25 05:18:57: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, waiting for 1 job, finished 1 job
2025-07-25 05:33:58: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #1/50
2025-07-25 05:34:37: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #2/50
2025-07-25 05:35:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #3/50
2025-07-25 05:35:56: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #4/50
2025-07-25 05:36:35: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #5/50
2025-07-25 05:37:13: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #6/50
2025-07-25 05:37:52: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #7/50
2025-07-25 05:38:32: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #8/50
2025-07-25 05:39:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #9/50
2025-07-25 05:39:50: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #10/50
2025-07-25 05:40:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #11/50
2025-07-25 05:41:07: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #12/50
2025-07-25 05:41:46: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #13/50
2025-07-25 05:42:25: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #14/50
2025-07-25 05:43:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #15/50
2025-07-25 05:43:44: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #16/50
2025-07-25 05:44:23: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #17/50
2025-07-25 05:45:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #18/50
2025-07-25 05:45:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #19/50
2025-07-25 05:46:23: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #20/50
2025-07-25 05:47:07: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #21/50
2025-07-25 05:47:46: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #22/50
2025-07-25 05:48:26: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #23/50
2025-07-25 05:49:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #24/50
2025-07-25 05:49:45: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #25/50
2025-07-25 05:50:24: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #26/50
2025-07-25 05:51:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #27/50
2025-07-25 05:51:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #28/50
2025-07-25 05:52:22: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #29/50
2025-07-25 05:53:01: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #30/50
2025-07-25 05:53:40: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #31/50
2025-07-25 05:54:19: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #32/50
2025-07-25 05:54:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #33/50
2025-07-25 05:55:39: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #34/50
2025-07-25 05:56:18: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #35/50
2025-07-25 05:56:58: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #36/50
2025-07-25 05:57:37: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #37/50
2025-07-25 05:58:16: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #38/50
2025-07-25 05:58:55: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #39/50
2025-07-25 05:59:34: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #40/50
2025-07-25 06:00:13: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #41/50
2025-07-25 06:00:52: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #42/50
2025-07-25 06:01:32: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #43/50
2025-07-25 06:02:13: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #44/50
2025-07-25 06:02:52: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #45/50
2025-07-25 06:03:32: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #46/50
2025-07-25 06:04:10: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #47/50
2025-07-25 06:04:51: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #48/50
2025-07-25 06:05:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #49/50
2025-07-25 06:06:09: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, getting new HP set #50/50
2025-07-25 06:06:49: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, requested 50 jobs, got 50, 56.87 s/job
2025-07-25 06:07:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #1/50 start
2025-07-25 06:07:28: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #2/50 start
2025-07-25 06:07:50: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #3/50 start
2025-07-25 06:08:12: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #4/50 start
2025-07-25 06:08:34: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #5/50 start
2025-07-25 06:08:55: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #6/50 start
2025-07-25 06:09:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #7/50 start
2025-07-25 06:09:39: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #8/50 start
2025-07-25 06:10:01: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #9/50 start
2025-07-25 06:10:23: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #10/50 start
2025-07-25 06:10:45: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #11/50 start
2025-07-25 06:11:07: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #12/50 start
2025-07-25 06:11:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #13/50 start
2025-07-25 06:11:51: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #14/50 start
2025-07-25 06:12:13: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #15/50 start
2025-07-25 06:12:35: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #16/50 start
2025-07-25 06:12:58: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #17/50 start
2025-07-25 06:13:21: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #18/50 start
2025-07-25 06:13:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #19/50 start
2025-07-25 06:14:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #20/50 start
2025-07-25 06:14:27: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #21/50 start
2025-07-25 06:14:49: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #22/50 start
2025-07-25 06:15:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #23/50 start
2025-07-25 06:15:33: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #24/50 start
2025-07-25 06:15:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #25/50 start
2025-07-25 06:16:21: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #26/50 start
2025-07-25 06:16:44: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #27/50 start
2025-07-25 06:17:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #28/50 start
2025-07-25 06:17:28: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #29/50 start
2025-07-25 06:17:49: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #30/50 start
2025-07-25 06:18:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #31/50 start
2025-07-25 06:18:33: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #32/50 start
2025-07-25 06:18:55: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #33/50 start
2025-07-25 06:19:22: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #34/50 start
2025-07-25 06:19:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #35/50 start
2025-07-25 06:20:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #36/50 start
2025-07-25 06:20:27: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #37/50 start
2025-07-25 06:20:49: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #38/50 start
2025-07-25 06:21:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #39/50 start
2025-07-25 06:21:33: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #40/50 start
2025-07-25 06:21:55: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #41/50 start
2025-07-25 06:22:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #42/50 start
2025-07-25 06:22:39: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #43/50 start
2025-07-25 06:23:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #44/50 start
2025-07-25 06:23:24: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #45/50 start
2025-07-25 06:23:45: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #46/50 start
2025-07-25 06:24:07: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #47/50 start
2025-07-25 06:24:30: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #48/50 start
2025-07-25 06:24:51: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #49/50 start
2025-07-25 06:25:14: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, eval #50/50 start
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:25:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:26:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:26:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:26:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:26:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, starting new job
2025-07-25 06:30:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 6 = ∑6/50, started new job
2025-07-25 06:30:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 7 = ∑7/50, started new job
2025-07-25 06:30:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 7 = ∑7/50, started new job
2025-07-25 06:30:01: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 11 = ∑11/50, started new job
2025-07-25 06:30:01: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 11 = ∑11/50, started new job
2025-07-25 06:30:01: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 11 = ∑11/50, started new job
2025-07-25 06:30:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, unknown 15 = ∑15/50, started new job
2025-07-25 06:30:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/1 = ∑15/50, started new job
2025-07-25 06:30:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/1 = ∑15/50, started new job
2025-07-25 06:30:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/1 = ∑15/50, started new job
2025-07-25 06:30:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/1 = ∑15/50, started new job
2025-07-25 06:30:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/1 = ∑15/50, started new job
2025-07-25 06:30:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/2 = ∑16/50, started new job
2025-07-25 06:30:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/2 = ∑16/50, started new job
2025-07-25 06:30:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/2 = ∑16/50, started new job
2025-07-25 06:30:08: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, pending/unknown 14/2 = ∑16/50, started new job
2025-07-25 06:34:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:34:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:34:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:34:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:34:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:34:41: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:35:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:35:09: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:35:14: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:35:16: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:35:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 15/1 = ∑16/50, starting new job
2025-07-25 06:38:36: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 13/3 = ∑16/50, starting new job
2025-07-25 06:38:37: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/5 = ∑21/50, started new job
2025-07-25 06:38:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/5 = ∑21/50, started new job
2025-07-25 06:38:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/5 = ∑21/50, started new job
2025-07-25 06:38:38: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/5 = ∑21/50, started new job
2025-07-25 06:38:40: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/6 = ∑22/50, starting new job
2025-07-25 06:38:40: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/5 = ∑21/50, started new job
2025-07-25 06:38:40: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 13/3/6 = ∑22/50, starting new job
2025-07-25 06:38:41: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 13/3/10 = ∑26/50, starting new job
2025-07-25 06:38:42: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:38:42: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, starting new job
2025-07-25 06:38:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:38:43: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:38:44: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:38:44: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:38:45: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 13/3/10/1 = ∑27/50, started new job
2025-07-25 06:43:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 24/3/5 = ∑32/50, started new job
2025-07-25 06:43:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 24/3/5 = ∑32/50, started new job
2025-07-25 06:43:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 24/3/5 = ∑32/50, started new job
2025-07-25 06:43:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 24/3/5 = ∑32/50, started new job
2025-07-25 06:43:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 24/3/5 = ∑32/50, started new job
2025-07-25 06:44:08: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:08: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:09: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:10: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:11: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:12: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:12: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:44:12: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 29/3 = ∑32/50, starting new job
2025-07-25 06:47:35: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 28/4 = ∑32/50, starting new job
2025-07-25 06:47:40: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 28/4 = ∑32/50, starting new job
2025-07-25 06:48:02: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/6 = ∑38/50, started new job
2025-07-25 06:48:03: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:04: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:05: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 28/4/9 = ∑41/50, started new job
2025-07-25 06:48:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 28/4/9 = ∑41/50, started new job
2025-07-25 06:51:27: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 36/5 = ∑41/50, starting new job
2025-07-25 06:51:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 36/5 = ∑41/50, starting new job
2025-07-25 06:51:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 36/5 = ∑41/50, starting new job
2025-07-25 06:51:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 36/5 = ∑41/50, starting new job
2025-07-25 06:51:29: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 36/5 = ∑41/50, starting new job
2025-07-25 06:51:31: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 36/5/2 = ∑43/50, started new job
2025-07-25 06:51:32: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 36/5/2 = ∑43/50, started new job
2025-07-25 06:56:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 38/5 = ∑43/50, starting new job
2025-07-25 06:56:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 38/5 = ∑43/50, starting new job
2025-07-25 06:57:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 38/5/5 = ∑48/50, started new job
2025-07-25 06:57:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 38/5/5 = ∑48/50, started new job
2025-07-25 06:57:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 38/5/5 = ∑48/50, started new job
2025-07-25 06:57:59: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/unknown 38/5/5 = ∑48/50, started new job
2025-07-25 06:58:00: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 38/5/5 = ∑48/50, started new job
2025-07-25 06:58:09: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending/unknown 38/5/5/2 = ∑50/50, started new job
2025-07-25 06:58:10: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed/pending 38/5/7 = ∑50/50, started new job
2025-07-25 07:00:25: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 98.09
2025-07-25 07:00:25: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 98.28
2025-07-25 07:00:25: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 38.92
2025-07-25 07:00:25: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 87.4
2025-07-25 07:00:26: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 98.06
2025-07-25 07:00:26: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 95.38
2025-07-25 07:00:27: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 43/7 = ∑50/50, new result: 95.85
2025-07-25 07:07:16: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 41/2 = ∑43/50, finishing jobs, finished 7 jobs
2025-07-25 07:07:37: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 41/2 = ∑43/50, new result: 98.28
2025-07-25 07:07:37: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running/completed 41/2 = ∑43/50, new result: 94.17
2025-07-25 07:09:15: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running 41 = ∑41/50, finishing previous jobs (43), finished 2 jobs
2025-07-25 07:09:39: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running 41 = ∑41/50, waiting for 41 jobs
2025-07-25 07:10:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running 41 = ∑41/50, new result: 98.07
2025-07-25 07:10:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running 41 = ∑41/50, new result: 87.35
2025-07-25 07:10:06: BoTorchGenerator, failed: 113, best VAL_ACC: 98.29, running 41 = ∑41/50, new result: 98.39
2025-07-25 07:12:36: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 38 = ∑38/50, waiting for 41 jobs, finished 3 jobs
2025-07-25 07:12:55: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 38 = ∑38/50, waiting for 38 jobs
2025-07-25 07:13:21: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 38 = ∑38/50, new result: 71.59
2025-07-25 07:13:21: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 38 = ∑38/50, new result: 98.02
2025-07-25 07:14:56: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 36 = ∑36/50, waiting for 38 jobs, finished 2 jobs
2025-07-25 07:15:17: BoTorchGenerator, failed: 113, best VAL_ACC: 98.39, running 36 = ∑36/50, waiting for 36 jobs
2025-07-25 07:15:42: BoTorchGenerator, failed: 113 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 36 = ∑36/50, job_failed
2025-07-25 07:16:28: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job
2025-07-25 07:16:48: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 35 = ∑35/50, waiting for 35 jobs
2025-07-25 07:17:14: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 35 = ∑35/50, new result: 98.36
2025-07-25 07:17:14: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 35 = ∑35/50, new result: 98.12
2025-07-25 07:18:49: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 33 = ∑33/50, waiting for 35 jobs, finished 2 jobs
2025-07-25 07:19:09: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 33 = ∑33/50, waiting for 33 jobs
2025-07-25 07:19:34: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 33 = ∑33/50, new result: 98.39
2025-07-25 07:20:44: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job
2025-07-25 07:21:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 07:21:29: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.39, running 32 = ∑32/50, new result: 98.42
2025-07-25 07:22:41: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job
2025-07-25 07:23:02: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:23:33: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:24:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:24:34: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:25:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:25:36: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:26:07: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:26:38: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:27:08: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, waiting for 31 jobs
2025-07-25 07:27:33: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, new result: 98.3
2025-07-25 07:28:44: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job
2025-07-25 07:29:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:29:34: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:30:06: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:30:36: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:31:07: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:31:38: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:32:08: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:32:38: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:33:09: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 07:33:36: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 30 = ∑30/50, new result: 98.33
2025-07-25 07:34:47: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 30 jobs, finished 1 job
2025-07-25 07:35:09: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:35:39: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:36:09: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:36:40: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:37:11: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:37:41: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:38:13: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:38:44: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:39:14: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 07:39:41: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 29 = ∑29/50, new result: 98.42
2025-07-25 07:41:11: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 28 = ∑28/50, waiting for 29 jobs, finished 1 job
2025-07-25 07:41:31: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 07:41:59: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 28 = ∑28/50, new result: 98.05
2025-07-25 07:43:10: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 28 jobs, finished 1 job
2025-07-25 07:43:30: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:44:00: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:44:31: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:45:01: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:45:32: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:46:03: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:46:34: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:47:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:47:35: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:48:06: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:48:37: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, waiting for 27 jobs
2025-07-25 07:49:02: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 27 = ∑27/50, new result: 97.66
2025-07-25 07:50:32: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 26 = ∑26/50, waiting for 27 jobs, finished 1 job
2025-07-25 07:50:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 26 = ∑26/50, waiting for 26 jobs
2025-07-25 07:51:17: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 26 = ∑26/50, new result: 47.37
2025-07-25 07:52:28: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 26 jobs, finished 1 job
2025-07-25 07:52:48: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:53:19: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:53:50: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:54:20: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:54:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:55:23: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:55:53: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:56:24: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:56:55: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:57:26: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:57:57: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:58:27: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:58:57: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:59:28: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 07:59:58: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:00:28: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:00:58: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:01:30: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:02:02: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:02:32: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:03:03: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:03:34: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:04:05: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:04:35: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:05:06: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:05:37: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:06:07: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:06:38: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:07:09: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:07:41: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:08:11: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:08:43: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:09:13: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:09:43: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:10:16: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:10:47: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:11:18: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:11:50: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, waiting for 25 jobs
2025-07-25 08:12:15: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 25 = ∑25/50, new result: 98.36
2025-07-25 08:13:26: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 24 = ∑24/50, waiting for 25 jobs, finished 1 job
2025-07-25 08:13:45: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 24 = ∑24/50, waiting for 24 jobs
2025-07-25 08:14:17: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 24 = ∑24/50, waiting for 24 jobs
2025-07-25 08:14:48: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 24 = ∑24/50, waiting for 24 jobs
2025-07-25 08:15:14: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 24 = ∑24/50, new result: 98.34
2025-07-25 08:16:25: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 24 jobs, finished 1 job
2025-07-25 08:16:47: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:17:17: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:17:47: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:18:18: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:18:48: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:19:20: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:19:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:20:21: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:20:52: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:21:24: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:22:00: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:22:33: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:23:05: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:23:36: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:24:08: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:24:39: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:25:11: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:25:42: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:26:15: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:26:46: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:27:16: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:27:49: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:28:24: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:28:55: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:29:26: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:29:57: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:30:32: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, waiting for 23 jobs
2025-07-25 08:30:59: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 23 = ∑23/50, new result: 97.79
2025-07-25 08:32:11: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 23 jobs, finished 1 job
2025-07-25 08:32:31: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:33:01: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:33:33: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:34:04: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:34:35: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:35:05: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:35:37: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:36:08: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:36:39: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:37:10: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 22 = ∑22/50, waiting for 22 jobs
2025-07-25 08:37:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:37:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:37:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:37:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:37:51: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:37:52: BoTorchGenerator, failed: 114 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 6/16 = ∑22/50, job_failed
2025-07-25 08:40:07: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 22 jobs, finished 6 jobs
2025-07-25 08:40:28: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:40:59: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:41:30: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:42:02: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:42:34: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:43:04: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:43:36: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:44:06: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:44:37: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:45:07: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:45:38: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:46:10: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:46:41: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:47:12: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:54: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:47:55: BoTorchGenerator, failed: 120 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 8/8 = ∑16/50, job_failed
2025-07-25 08:50:25: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 16 jobs, finished 8 jobs
2025-07-25 08:50:45: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:51:16: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:51:46: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:52:17: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:52:49: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:53:20: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:53:51: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:54:22: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:54:52: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:55:23: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:55:53: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:56:25: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:56:56: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:57:28: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 8 = ∑8/50, waiting for 8 jobs
2025-07-25 08:58:09: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 4/4 = ∑8/50, job_failed
2025-07-25 08:58:09: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 4/4 = ∑8/50, job_failed
2025-07-25 08:58:09: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 4/4 = ∑8/50, job_failed
2025-07-25 08:58:09: BoTorchGenerator, failed: 128 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout/running 4/4 = ∑8/50, job_failed
2025-07-25 08:59:49: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 8 jobs, finished 4 jobs
2025-07-25 09:00:10: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:00:40: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:01:10: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:01:41: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:02:12: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:02:43: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:03:13: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:03:44: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:04:14: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:04:44: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:05:14: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:05:44: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:06:14: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:06:44: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:07:15: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:07:46: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 4 = ∑4/50, waiting for 4 jobs
2025-07-25 09:08:27: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout 4 = ∑4/50, job_failed
2025-07-25 09:08:27: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout 4 = ∑4/50, job_failed
2025-07-25 09:08:27: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout 4 = ∑4/50, job_failed
2025-07-25 09:08:27: BoTorchGenerator, failed: 132 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, timeout 4 = ∑4/50, job_failed
2025-07-25 09:09:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, waiting for 4 jobs, finished 4 jobs
2025-07-25 09:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #1/50
2025-07-25 09:27:25: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #2/50
2025-07-25 09:28:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #3/50
2025-07-25 09:28:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #4/50
2025-07-25 09:29:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #5/50
2025-07-25 09:30:20: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #6/50
2025-07-25 09:31:06: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #7/50
2025-07-25 09:31:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #7/50
2025-07-25 09:32:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #8/50
2025-07-25 09:32:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #9/50
2025-07-25 09:33:14: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #9/50
2025-07-25 09:33:59: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #10/50
2025-07-25 09:34:44: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #11/50
2025-07-25 09:35:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #12/50
2025-07-25 09:36:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #13/50
2025-07-25 09:36:56: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #14/50
2025-07-25 09:37:40: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #15/50
2025-07-25 09:38:24: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #16/50
2025-07-25 09:39:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #17/50
2025-07-25 09:39:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #18/50
2025-07-25 09:40:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #19/50
2025-07-25 09:41:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #20/50
2025-07-25 09:42:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #21/50
2025-07-25 09:42:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #22/50
2025-07-25 09:43:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #23/50
2025-07-25 09:44:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #24/50
2025-07-25 09:45:06: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #25/50
2025-07-25 09:45:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #26/50
2025-07-25 09:46:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #27/50
2025-07-25 09:47:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #28/50
2025-07-25 09:48:05: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #29/50
2025-07-25 09:48:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #30/50
2025-07-25 09:49:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #31/50
2025-07-25 09:50:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #32/50
2025-07-25 09:51:05: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #33/50
2025-07-25 09:51:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #34/50
2025-07-25 09:52:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #35/50
2025-07-25 09:53:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #36/50
2025-07-25 09:54:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #37/50
2025-07-25 09:54:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #38/50
2025-07-25 09:55:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #39/50
2025-07-25 09:56:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #40/50
2025-07-25 09:57:05: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #41/50
2025-07-25 09:57:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #42/50
2025-07-25 09:58:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #43/50
2025-07-25 09:59:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #44/50
2025-07-25 10:00:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #45/50
2025-07-25 10:00:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #46/50
2025-07-25 10:01:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #47/50
2025-07-25 10:02:20: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #48/50
2025-07-25 10:03:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #49/50
2025-07-25 10:04:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, getting new HP set #50/50
2025-07-25 10:05:23: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, requested 50 jobs, got 50, 65.96 s/job
2025-07-25 10:05:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #1/50 start
2025-07-25 10:06:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #2/50 start
2025-07-25 10:06:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #3/50 start
2025-07-25 10:06:59: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #4/50 start
2025-07-25 10:07:24: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #5/50 start
2025-07-25 10:07:49: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #6/50 start
2025-07-25 10:08:14: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #7/50 start
2025-07-25 10:08:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #8/50 start
2025-07-25 10:09:03: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #9/50 start
2025-07-25 10:09:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #10/50 start
2025-07-25 10:09:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #11/50 start
2025-07-25 10:10:18: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #12/50 start
2025-07-25 10:10:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #13/50 start
2025-07-25 10:11:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #14/50 start
2025-07-25 10:11:32: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #15/50 start
2025-07-25 10:11:57: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #16/50 start
2025-07-25 10:12:23: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #17/50 start
2025-07-25 10:12:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #18/50 start
2025-07-25 10:13:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #19/50 start
2025-07-25 10:13:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #20/50 start
2025-07-25 10:14:02: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #21/50 start
2025-07-25 10:14:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #22/50 start
2025-07-25 10:14:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #23/50 start
2025-07-25 10:15:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #24/50 start
2025-07-25 10:15:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #25/50 start
2025-07-25 10:16:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #26/50 start
2025-07-25 10:16:31: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #27/50 start
2025-07-25 10:16:57: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #28/50 start
2025-07-25 10:17:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #29/50 start
2025-07-25 10:17:47: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #30/50 start
2025-07-25 10:18:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #31/50 start
2025-07-25 10:18:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #32/50 start
2025-07-25 10:19:04: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #33/50 start
2025-07-25 10:19:29: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #34/50 start
2025-07-25 10:19:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #35/50 start
2025-07-25 10:20:19: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #36/50 start
2025-07-25 10:20:45: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #37/50 start
2025-07-25 10:21:11: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #38/50 start
2025-07-25 10:21:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #39/50 start
2025-07-25 10:22:01: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #40/50 start
2025-07-25 10:22:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #41/50 start
2025-07-25 10:22:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #42/50 start
2025-07-25 10:23:16: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #43/50 start
2025-07-25 10:23:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #44/50 start
2025-07-25 10:24:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #45/50 start
2025-07-25 10:24:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #46/50 start
2025-07-25 10:24:58: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #47/50 start
2025-07-25 10:25:23: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #48/50 start
2025-07-25 10:25:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #49/50 start
2025-07-25 10:26:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, eval #50/50 start
2025-07-25 10:26:40: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:26:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, starting new job
2025-07-25 10:30:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, unknown 2 = ∑2/50, started new job
2025-07-25 10:30:40: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, unknown 2 = ∑2/50, started new job
2025-07-25 10:30:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/6 = ∑8/50, started new job
2025-07-25 10:30:47: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/9 = ∑11/50, started new job
2025-07-25 10:30:47: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/9 = ∑11/50, started new job
2025-07-25 10:30:47: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/10 = ∑12/50, started new job
2025-07-25 10:30:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/14 = ∑16/50, started new job
2025-07-25 10:30:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/14 = ∑16/50, started new job
2025-07-25 10:30:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/12 = ∑14/50, started new job
2025-07-25 10:30:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/14 = ∑16/50, started new job
2025-07-25 10:30:49: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, pending/unknown 2/14 = ∑16/50, started new job
2025-07-25 10:30:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 10/6 = ∑16/50, started new job
2025-07-25 10:30:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 10/6 = ∑16/50, started new job
2025-07-25 10:30:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 10/6 = ∑16/50, started new job
2025-07-25 10:30:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 10/6 = ∑16/50, started new job
2025-07-25 10:30:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 10/6 = ∑16/50, started new job
2025-07-25 10:35:15: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:35:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:35:30: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:35:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:35:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:35:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:23: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:25: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:31: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:36:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 16 = ∑16/50, starting new job
2025-07-25 10:38:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 16/1 = ∑17/50, started new job
2025-07-25 10:39:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 17/1 = ∑18/50, started new job
2025-07-25 10:39:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 17/1/1 = ∑19/50, started new job
2025-07-25 10:39:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 19/3 = ∑22/50, started new job
2025-07-25 10:39:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 19/3 = ∑22/50, started new job
2025-07-25 10:39:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 19/3 = ∑22/50, started new job
2025-07-25 10:40:32: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 22/2 = ∑24/50, started new job
2025-07-25 10:40:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 22/2 = ∑24/50, started new job
2025-07-25 10:40:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 22/2/1 = ∑25/50, started new job
2025-07-25 10:40:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 22/3/3 = ∑28/50, started new job
2025-07-25 10:40:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 22/3/3 = ∑28/50, started new job
2025-07-25 10:40:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 22/3/3 = ∑28/50, started new job
2025-07-25 10:40:44: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 22/3/6 = ∑31/50, starting new job
2025-07-25 10:40:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 22/9 = ∑31/50, started new job
2025-07-25 10:40:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 22/9 = ∑31/50, started new job
2025-07-25 10:40:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 22/9 = ∑31/50, started new job
2025-07-25 10:43:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:44:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:44:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:44:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:44:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:44:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 31 = ∑31/50, starting new job
2025-07-25 10:45:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 31/1 = ∑32/50, started new job
2025-07-25 10:46:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:32: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:46:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 32 = ∑32/50, starting new job
2025-07-25 10:47:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 32/1 = ∑33/50, started new job
2025-07-25 10:48:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 33/2 = ∑35/50, started new job
2025-07-25 10:48:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 33/4 = ∑37/50, started new job
2025-07-25 10:48:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 33/4 = ∑37/50, started new job
2025-07-25 10:48:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 33/4 = ∑37/50, started new job
2025-07-25 10:48:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 33/4/1 = ∑38/50, started new job
2025-07-25 10:50:04: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 38 = ∑38/50, starting new job
2025-07-25 10:50:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 38/2 = ∑40/50, started new job
2025-07-25 10:50:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/1/1 = ∑40/50, started new job
2025-07-25 10:50:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/2/2 = ∑42/50, started new job
2025-07-25 10:50:47: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/2/3 = ∑43/50, started new job
2025-07-25 10:50:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending 38/5 = ∑43/50, started new job
2025-07-25 10:50:56: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/5/3 = ∑46/50, started new job
2025-07-25 10:50:56: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/5/3 = ∑46/50, started new job
2025-07-25 10:50:56: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/pending/unknown 38/5/3 = ∑46/50, started new job
2025-07-25 10:51:06: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 46/1 = ∑47/50, started new job
2025-07-25 10:52:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, starting new job
2025-07-25 10:54:16: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, starting new job
2025-07-25 10:54:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 47/1 = ∑48/50, started new job
2025-07-25 10:55:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 48/1 = ∑49/50, started new job
2025-07-25 10:56:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running/unknown 49/1 = ∑50/50, started new job
2025-07-25 10:57:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 10:57:55: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 10:58:30: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 10:59:04: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 10:59:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:00:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:00:45: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:01:19: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:01:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:02:32: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:03:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:03:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:04:18: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:04:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:05:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:06:06: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:06:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:07:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:07:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:08:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:09:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:09:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:10:30: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:11:05: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:11:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:12:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:12:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:13:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:14:02: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:14:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:15:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:15:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:16:44: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:18:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:19:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:20:23: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:21:00: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:21:36: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:22:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:22:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:23:16: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 11:23:44: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 50 = ∑50/50, new result: 98.28
2025-07-25 11:25:04: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job
2025-07-25 11:25:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:26:01: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:26:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:27:08: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:27:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:28:18: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:28:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:29:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:30:05: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:30:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:31:13: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:31:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:32:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:32:58: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:33:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:34:08: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:34:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:35:18: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:35:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:36:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:37:00: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:37:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:38:08: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:38:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:39:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:39:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:40:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:41:02: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:41:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:42:11: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 11:42:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 49 = ∑49/50, new result: 98.36
2025-07-25 11:44:02: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-25 11:44:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:45:02: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:45:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:46:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:46:48: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:47:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:47:56: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:48:30: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:49:03: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:49:37: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:50:11: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:50:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:51:16: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:51:49: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:52:22: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:52:57: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:53:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:54:10: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:54:46: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:55:21: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:55:55: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:56:28: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:57:03: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:57:40: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:58:15: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:58:50: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 11:59:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:00:00: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:00:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:01:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:01:45: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:02:20: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:02:57: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:03:33: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 12:04:03: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 48 = ∑48/50, new result: 98.3
2025-07-25 12:05:30: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-25 12:05:57: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 12:06:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 12:07:12: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 12:07:45: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 47 = ∑47/50, new result: 10.29
2025-07-25 12:09:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-25 12:10:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 12:10:38: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 46 = ∑46/50, new result: 98.45
2025-07-25 12:10:39: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.42, running 46 = ∑46/50, new result: 98.37
2025-07-25 12:12:31: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 46 jobs, finished 2 jobs
2025-07-25 12:12:55: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:13:31: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:14:07: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:14:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:15:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:15:52: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:16:25: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:16:59: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:17:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:18:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:18:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:19:17: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:19:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:20:26: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:21:01: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:21:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:22:08: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:22:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:23:18: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:23:53: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:24:27: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:25:00: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:25:35: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:26:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:26:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:27:16: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:27:51: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:28:25: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:29:00: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:29:34: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:30:09: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:30:43: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:31:19: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:31:54: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 44 = ∑44/50, waiting for 44 jobs
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:41: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:42: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:32:44: BoTorchGenerator, failed: 136 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 15/29 = ∑44/50, job_failed
2025-07-25 12:37:56: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 29 = ∑29/50, waiting for 44 jobs, finished 15 jobs
2025-07-25 12:38:27: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 29 = ∑29/50, waiting for 29 jobs
2025-07-25 12:38:56: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 29 = ∑29/50, new result: 9.82
2025-07-25 12:40:19: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 28 = ∑28/50, waiting for 29 jobs, finished 1 job
2025-07-25 12:40:41: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 12:41:15: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 12:41:51: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 28 = ∑28/50, waiting for 28 jobs
2025-07-25 12:42:27: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:43:01: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:43:34: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:44:08: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:44:41: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:45:15: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:45:50: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:46:22: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:47:57: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:51:08: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:51:45: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:52:21: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 12:53:06: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:06: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:06: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:06: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:07: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:08: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 12:53:09: BoTorchGenerator, failed: 151 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout/running 13/2 = ∑15/50, job_failed
2025-07-25 13:00:18: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 15 jobs, finished 13 jobs
2025-07-25 13:00:43: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 2 jobs
2025-07-25 13:01:30: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 2 jobs
2025-07-25 13:02:04: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 2 jobs
2025-07-25 13:02:37: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 2 = ∑2/50, waiting for 2 jobs
2025-07-25 13:03:21: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout 2 = ∑2/50, job_failed
2025-07-25 13:03:21: BoTorchGenerator, failed: 164 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, timeout 2 = ∑2/50, job_failed
2025-07-25 13:04:23: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, waiting for 2 jobs, finished 2 jobs
2025-07-25 13:24:01: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #1/50
2025-07-25 13:24:51: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #2/50
2025-07-25 13:25:41: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #3/50
2025-07-25 13:26:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #4/50
2025-07-25 13:27:23: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #5/50
2025-07-25 13:28:14: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #6/50
2025-07-25 13:29:03: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #7/50
2025-07-25 13:29:52: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #8/50
2025-07-25 13:30:41: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #9/50
2025-07-25 13:31:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #10/50
2025-07-25 13:32:19: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #11/50
2025-07-25 13:33:08: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #12/50
2025-07-25 13:34:00: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #13/50
2025-07-25 13:34:51: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #14/50
2025-07-25 13:35:14: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #14/50
2025-07-25 13:36:05: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #15/50
2025-07-25 13:36:54: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #16/50
2025-07-25 13:37:16: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #16/50
2025-07-25 13:38:05: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #17/50
2025-07-25 13:38:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #18/50
2025-07-25 13:39:45: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #19/50
2025-07-25 13:40:34: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #20/50
2025-07-25 13:41:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #21/50
2025-07-25 13:42:24: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #22/50
2025-07-25 13:43:17: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #23/50
2025-07-25 13:44:11: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #24/50
2025-07-25 13:45:02: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #25/50
2025-07-25 13:45:53: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #26/50
2025-07-25 13:46:43: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #27/50
2025-07-25 13:47:33: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #28/50
2025-07-25 13:48:22: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #29/50
2025-07-25 13:49:11: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #30/50
2025-07-25 13:50:02: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #31/50
2025-07-25 13:50:51: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #32/50
2025-07-25 13:51:40: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #33/50
2025-07-25 13:52:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #34/50
2025-07-25 13:53:20: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #35/50
2025-07-25 13:54:10: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #36/50
2025-07-25 13:55:01: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #37/50
2025-07-25 13:55:51: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #38/50
2025-07-25 13:56:40: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #39/50
2025-07-25 13:57:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #40/50
2025-07-25 13:58:18: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #41/50
2025-07-25 13:59:08: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #42/50
2025-07-25 14:00:04: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #43/50
2025-07-25 14:00:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #44/50
2025-07-25 14:01:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #45/50
2025-07-25 14:02:33: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #46/50
2025-07-25 14:03:23: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #47/50
2025-07-25 14:04:12: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #48/50
2025-07-25 14:05:46: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #49/50
2025-07-25 14:06:35: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, getting new HP set #50/50
2025-07-25 14:07:24: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, requested 50 jobs, got 50, 74.95 s/job
2025-07-25 14:07:46: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #1/50 start
2025-07-25 14:08:13: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #2/50 start
2025-07-25 14:08:40: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #3/50 start
2025-07-25 14:09:07: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #4/50 start
2025-07-25 14:09:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #5/50 start
2025-07-25 14:10:04: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #6/50 start
2025-07-25 14:10:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #7/50 start
2025-07-25 14:10:58: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #8/50 start
2025-07-25 14:11:26: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #9/50 start
2025-07-25 14:11:53: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #10/50 start
2025-07-25 14:12:21: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #11/50 start
2025-07-25 14:12:49: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #12/50 start
2025-07-25 14:13:17: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #13/50 start
2025-07-25 14:13:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #14/50 start
2025-07-25 14:14:11: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #15/50 start
2025-07-25 14:14:38: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #16/50 start
2025-07-25 14:15:05: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #17/50 start
2025-07-25 14:15:32: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #18/50 start
2025-07-25 14:16:01: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #19/50 start
2025-07-25 14:16:28: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #20/50 start
2025-07-25 14:16:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #21/50 start
2025-07-25 14:17:22: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #22/50 start
2025-07-25 14:17:49: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #23/50 start
2025-07-25 14:18:17: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #24/50 start
2025-07-25 14:18:43: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #25/50 start
2025-07-25 14:19:11: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #26/50 start
2025-07-25 14:19:38: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #27/50 start
2025-07-25 14:20:05: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #28/50 start
2025-07-25 14:20:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #29/50 start
2025-07-25 14:20:58: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #30/50 start
2025-07-25 14:21:24: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #31/50 start
2025-07-25 14:21:50: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #32/50 start
2025-07-25 14:22:17: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #33/50 start
2025-07-25 14:22:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #34/50 start
2025-07-25 14:23:10: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #35/50 start
2025-07-25 14:23:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #36/50 start
2025-07-25 14:24:03: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #37/50 start
2025-07-25 14:24:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #38/50 start
2025-07-25 14:25:14: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #39/50 start
2025-07-25 14:25:42: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #40/50 start
2025-07-25 14:26:10: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #41/50 start
2025-07-25 14:26:42: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #42/50 start
2025-07-25 14:27:10: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #43/50 start
2025-07-25 14:27:38: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #44/50 start
2025-07-25 14:28:06: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #45/50 start
2025-07-25 14:28:34: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #46/50 start
2025-07-25 14:29:02: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #47/50 start
2025-07-25 14:29:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #48/50 start
2025-07-25 14:29:58: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #49/50 start
2025-07-25 14:30:26: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, eval #50/50 start
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:30:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, starting new job
2025-07-25 14:35:15: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, unknown 1 = ∑1/50, started new job
2025-07-25 14:35:22: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 1/2 = ∑3/50, started new job
2025-07-25 14:35:23: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 1/2 = ∑3/50, started new job
2025-07-25 14:35:27: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/4 = ∑7/50, started new job
2025-07-25 14:35:28: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/9 = ∑12/50, started new job
2025-07-25 14:35:28: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/11 = ∑14/50, started new job
2025-07-25 14:35:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/12 = ∑15/50, started new job
2025-07-25 14:35:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending/unknown 3/13 = ∑16/50, started new job
2025-07-25 14:35:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending 16 = ∑16/50, started new job
2025-07-25 14:35:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending 16 = ∑16/50, started new job
2025-07-25 14:35:31: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, pending 16 = ∑16/50, started new job
2025-07-25 14:40:49: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:53: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:53: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:54: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:54: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:55: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:40:56: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:07: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:19: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:46: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:50: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:51: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:52: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:52: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:41:53: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 15/1 = ∑16/50, starting new job
2025-07-25 14:45:19: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 15/1/1 = ∑17/50, started new job
2025-07-25 14:45:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 16/1/1 = ∑18/50, started new job
2025-07-25 14:45:33: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 16/1/1/2 = ∑20/50, started new job
2025-07-25 14:45:34: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 16/1/1/4 = ∑22/50, started new job
2025-07-25 14:45:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 16/1/5 = ∑22/50, started new job
2025-07-25 14:45:37: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 16/1/5 = ∑22/50, started new job
2025-07-25 14:45:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 16/1/5/3 = ∑25/50, started new job
2025-07-25 14:45:45: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 16/1/5/3 = ∑25/50, started new job
2025-07-25 14:45:47: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 16/1/8 = ∑25/50, started new job
2025-07-25 14:46:03: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 24/1/1 = ∑26/50, started new job
2025-07-25 14:46:26: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown/pending 25/1/1/1 = ∑28/50, started new job
2025-07-25 14:46:27: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown/pending 25/1/1/1 = ∑28/50, started new job
2025-07-25 14:46:35: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 25/1/2/4 = ∑32/50, started new job
2025-07-25 14:46:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 25/1/6 = ∑32/50, started new job
2025-07-25 14:46:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 25/1/6 = ∑32/50, started new job
2025-07-25 14:46:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 25/1/6 = ∑32/50, started new job
2025-07-25 14:51:12: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:51:29: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:51:40: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:03: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:06: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:16: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:25: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:25: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:35: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:35: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:35: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:36: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:52:37: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 14:53:30: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 31/1/1 = ∑33/50, started new job
2025-07-25 14:55:34: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 32/1/1 = ∑34/50, started new job
2025-07-25 14:56:09: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 33/1/1 = ∑35/50, started new job
2025-07-25 14:56:44: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 34/1/2 = ∑37/50, started new job
2025-07-25 14:56:45: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 34/1/2 = ∑37/50, started new job
2025-07-25 14:57:05: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/unknown 36/1/1 = ∑38/50, started new job
2025-07-25 14:57:10: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/1/1 = ∑39/50, started new job
2025-07-25 14:57:13: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/4/1 = ∑42/50, started new job
2025-07-25 14:57:13: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/4/1 = ∑42/50, started new job
2025-07-25 14:57:14: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/4/1 = ∑42/50, started new job
2025-07-25 14:57:20: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/6/4 = ∑47/50, started new job
2025-07-25 14:57:20: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/6/4 = ∑47/50, started new job
2025-07-25 14:57:20: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/6/4 = ∑47/50, started new job
2025-07-25 14:57:21: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/6/4 = ∑47/50, started new job
2025-07-25 14:57:22: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/6/4 = ∑47/50, started new job
2025-07-25 14:57:25: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending/unknown 36/1/10/1 = ∑48/50, started new job
2025-07-25 14:58:54: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 47/1 = ∑48/50, starting new job
2025-07-25 15:01:06: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 47/1 = ∑48/50, starting new job
2025-07-25 15:02:54: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 47/1/1 = ∑49/50, started new job
2025-07-25 15:03:23: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory/pending 48/1/1 = ∑50/50, started new job
2025-07-25 15:04:40: BoTorchGenerator, failed: 166 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running/out_of_memory 49/1 = ∑50/50, job_failed
2025-07-25 15:05:38: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, finishing jobs, finished 1 job
2025-07-25 15:06:17: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:06:55: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:07:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:08:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:08:46: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:09:24: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:10:03: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:10:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:11:19: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:11:55: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:12:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:13:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:13:46: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:14:24: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:15:01: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:15:38: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:16:15: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:16:52: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:17:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:18:06: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:18:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:19:20: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:19:56: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:20:33: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:21:10: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:21:49: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:22:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:23:04: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:23:41: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:24:17: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:24:54: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:25:31: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:26:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:26:46: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:27:24: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:28:02: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:28:39: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:29:15: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:29:55: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:30:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:31:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:31:47: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:32:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:33:10: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:33:51: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:34:31: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:35:08: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:35:45: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:36:22: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:36:58: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:37:35: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:38:12: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:38:49: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:39:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:40:04: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:40:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:41:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:42:10: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:42:51: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:43:34: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:44:12: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:44:49: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:45:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:46:05: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:46:45: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 15:47:20: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 49 = ∑49/50, new result: 98.42
2025-07-25 15:48:50: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-25 15:49:16: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:49:52: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:50:29: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:51:07: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:51:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:52:19: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:52:55: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:53:33: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:54:10: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:54:48: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:55:25: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:56:04: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:56:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:57:20: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:58:34: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:59:12: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 15:59:50: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:00:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:01:05: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:01:42: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:02:21: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:02:59: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:03:37: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:04:15: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:04:54: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:05:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:06:22: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:07:01: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 16:07:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 48 = ∑48/50, new result: 98.35
2025-07-25 16:09:02: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-25 16:09:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 16:10:06: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 16:10:45: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 47 = ∑47/50, waiting for 47 jobs
2025-07-25 16:11:15: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 47 = ∑47/50, new result: 98.37
2025-07-25 16:12:43: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-25 16:13:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:13:46: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:14:23: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:15:02: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:15:38: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:16:16: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:16:52: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:17:29: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:18:10: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:18:48: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:19:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:20:02: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:20:39: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:21:16: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:21:54: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:22:31: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:23:08: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:23:45: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:24:22: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:24:59: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:25:37: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:26:14: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:26:52: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:27:30: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:28:07: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, waiting for 46 jobs
2025-07-25 16:28:40: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 46 = ∑46/50, new result: 97.72
2025-07-25 16:30:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job
2025-07-25 16:30:34: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:31:12: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:31:49: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:32:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:33:02: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:33:39: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:34:16: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:34:53: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:35:31: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:36:09: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:36:47: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 45 = ∑45/50, waiting for 45 jobs
2025-07-25 16:37:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:38:04: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:38:42: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:39:19: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:39:56: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:40:33: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:41:13: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:41:51: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:42:27: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:43:03: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:43:42: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:44:18: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:44:56: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:45:36: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:46:14: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:46:52: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 30 = ∑30/50, waiting for 30 jobs
2025-07-25 16:47:30: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 16:48:01: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.45, running 16 = ∑16/50, new result: 98.47
2025-07-25 16:49:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 16 jobs, finished 1 job
2025-07-25 16:49:58: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:50:48: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:51:26: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:52:04: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:52:41: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:53:19: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:53:56: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:54:32: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:55:11: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:55:50: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:56:28: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:57:07: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 15 = ∑15/50, waiting for 15 jobs
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:57: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 16:57:58: BoTorchGenerator, failed: 167 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 12/3 = ∑15/50, job_failed
2025-07-25 17:02:37: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 15 jobs, finished 12 jobs
2025-07-25 17:03:05: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:03:42: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:04:19: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:04:54: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:05:30: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:06:07: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:06:43: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:07:20: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 3 = ∑3/50, waiting for 3 jobs
2025-07-25 17:08:04: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 3 = ∑3/50, job_failed
2025-07-25 17:08:05: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 3 = ∑3/50, job_failed
2025-07-25 17:08:05: BoTorchGenerator, failed: 179 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 3 = ∑3/50, job_failed
2025-07-25 17:09:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, waiting for 3 jobs, finished 3 jobs
2025-07-25 17:28:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #1/50
2025-07-25 17:29:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #2/50
2025-07-25 17:30:29: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #3/50
2025-07-25 17:31:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #4/50
2025-07-25 17:31:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #4/50
2025-07-25 17:32:41: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #5/50
2025-07-25 17:33:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #6/50
2025-07-25 17:34:28: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #7/50
2025-07-25 17:34:52: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #7/50
2025-07-25 17:35:46: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #8/50
2025-07-25 17:36:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-25 17:37:04: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-25 17:37:58: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50
2025-07-25 17:38:52: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #11/50
2025-07-25 17:39:46: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #12/50
2025-07-25 17:40:10: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #12/50
2025-07-25 17:41:05: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #13/50
2025-07-25 17:41:58: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #14/50
2025-07-25 17:42:51: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #15/50
2025-07-25 17:43:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #16/50
2025-07-25 17:44:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #17/50
2025-07-25 17:45:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50
2025-07-25 17:46:26: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #19/50
2025-07-25 17:47:20: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #20/50
2025-07-25 17:48:14: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #21/50
2025-07-25 17:49:08: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #22/50
2025-07-25 17:50:01: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #23/50
2025-07-25 17:50:55: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #24/50
2025-07-25 17:51:49: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #25/50
2025-07-25 17:52:42: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #26/50
2025-07-25 17:53:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #27/50
2025-07-25 17:54:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #28/50
2025-07-25 17:55:25: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #29/50
2025-07-25 17:56:19: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #30/50
2025-07-25 17:57:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #31/50
2025-07-25 17:58:07: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #32/50
2025-07-25 17:59:01: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #33/50
2025-07-25 17:59:56: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #34/50
2025-07-25 18:00:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50
2025-07-25 18:01:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50
2025-07-25 18:02:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #36/50
2025-07-25 18:03:03: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #37/50
2025-07-25 18:03:28: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #37/50
2025-07-25 18:04:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #38/50
2025-07-25 18:05:17: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #39/50
2025-07-25 18:06:11: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #40/50
2025-07-25 18:07:06: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #41/50
2025-07-25 18:08:00: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #42/50
2025-07-25 18:08:55: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #43/50
2025-07-25 18:09:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #44/50
2025-07-25 18:12:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #45/50
2025-07-25 18:13:56: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #46/50
2025-07-25 18:14:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #47/50
2025-07-25 18:15:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #48/50
2025-07-25 18:16:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #49/50
2025-07-25 18:17:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #50/50
2025-07-25 18:19:21: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #50/50
2025-07-25 18:20:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, requested 50 jobs, got 50, 84.33 s/job
2025-07-25 18:20:51: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #1/50 start
2025-07-25 18:21:29: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #2/50 start
2025-07-25 18:22:06: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #3/50 start
2025-07-25 18:22:38: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #4/50 start
2025-07-25 18:23:08: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #5/50 start
2025-07-25 18:23:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #6/50 start
2025-07-25 18:24:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #7/50 start
2025-07-25 18:24:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #8/50 start
2025-07-25 18:25:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #9/50 start
2025-07-25 18:25:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #10/50 start
2025-07-25 18:26:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #11/50 start
2025-07-25 18:26:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #12/50 start
2025-07-25 18:27:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #13/50 start
2025-07-25 18:27:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #14/50 start
2025-07-25 18:28:11: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #15/50 start
2025-07-25 18:28:41: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #16/50 start
2025-07-25 18:29:12: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #17/50 start
2025-07-25 18:29:42: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #18/50 start
2025-07-25 18:30:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #19/50 start
2025-07-25 18:30:44: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #20/50 start
2025-07-25 18:31:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #21/50 start
2025-07-25 18:31:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #22/50 start
2025-07-25 18:32:14: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #23/50 start
2025-07-25 18:32:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #24/50 start
2025-07-25 18:33:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #25/50 start
2025-07-25 18:33:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #26/50 start
2025-07-25 18:34:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #27/50 start
2025-07-25 18:34:46: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #28/50 start
2025-07-25 18:35:16: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #29/50 start
2025-07-25 18:35:46: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #30/50 start
2025-07-25 18:36:16: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #31/50 start
2025-07-25 18:36:46: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #32/50 start
2025-07-25 18:37:16: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #33/50 start
2025-07-25 18:37:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #34/50 start
2025-07-25 18:38:17: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #35/50 start
2025-07-25 18:38:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #36/50 start
2025-07-25 18:39:17: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #37/50 start
2025-07-25 18:39:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #38/50 start
2025-07-25 18:40:17: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #39/50 start
2025-07-25 18:40:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #40/50 start
2025-07-25 18:41:18: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #41/50 start
2025-07-25 18:41:48: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #42/50 start
2025-07-25 18:42:18: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #43/50 start
2025-07-25 18:42:48: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #44/50 start
2025-07-25 18:43:19: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #45/50 start
2025-07-25 18:43:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #46/50 start
2025-07-25 18:44:20: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #47/50 start
2025-07-25 18:44:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #48/50 start
2025-07-25 18:45:20: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #49/50 start
2025-07-25 18:45:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #50/50 start
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:23: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:24: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:24: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:46:24: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 18:51:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 5 = ∑5/50, started new job
2025-07-25 18:51:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 5 = ∑5/50, started new job
2025-07-25 18:51:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 5 = ∑5/50, started new job
2025-07-25 18:51:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 6 = ∑6/50, started new job
2025-07-25 18:51:31: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 8 = ∑8/50, started new job
2025-07-25 18:51:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/3 = ∑11/50, started new job
2025-07-25 18:51:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/4 = ∑12/50, started new job
2025-07-25 18:51:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:36: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:51:38: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 8/8 = ∑16/50, started new job
2025-07-25 18:57:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:36: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:38: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:41: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:57:44: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:06: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:21: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:36: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:36: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 18:58:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 19:03:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 16/2 = ∑18/50, started new job
2025-07-25 19:03:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 16/2 = ∑18/50, started new job
2025-07-25 19:03:38: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 16/2/3 = ∑21/50, started new job
2025-07-25 19:03:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 16/2/4 = ∑22/50, started new job
2025-07-25 19:03:39: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 16/2/4 = ∑22/50, started new job
2025-07-25 19:03:41: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 24/1 = ∑25/50, started new job
2025-07-25 19:03:42: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 24/1 = ∑25/50, started new job
2025-07-25 19:03:43: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 24/1 = ∑25/50, started new job
2025-07-25 19:03:43: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 24/1 = ∑25/50, started new job
2025-07-25 19:04:01: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 24/1/1 = ∑26/50, started new job
2025-07-25 19:04:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 24/2/1 = ∑27/50, started new job
2025-07-25 19:05:48: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 27/1/4 = ∑32/50, started new job
2025-07-25 19:05:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 27/1/4 = ∑32/50, started new job
2025-07-25 19:05:51: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 27/1/4 = ∑32/50, started new job
2025-07-25 19:05:51: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 27/1/4 = ∑32/50, started new job
2025-07-25 19:05:51: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 27/1/4 = ∑32/50, started new job
2025-07-25 19:10:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:10:40: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:10:45: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:10:59: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:17: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:24: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:26: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:28: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:29: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:32: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:32: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:11:32: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-25 19:15:14: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 32/1 = ∑33/50, started new job
2025-07-25 19:15:22: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 33/1 = ∑34/50, started new job
2025-07-25 19:15:52: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 35 = ∑35/50, started new job
2025-07-25 19:15:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 35/1 = ∑36/50, started new job
2025-07-25 19:16:41: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 36/1 = ∑37/50, started new job
2025-07-25 19:16:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 36/1/2 = ∑39/50, started new job
2025-07-25 19:16:50: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 36/1/2 = ∑39/50, started new job
2025-07-25 19:16:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 39/5 = ∑44/50, started new job
2025-07-25 19:16:55: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 39/8 = ∑47/50, started new job
2025-07-25 19:16:56: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 39/8 = ∑47/50, started new job
2025-07-25 19:16:56: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 39/8 = ∑47/50, started new job
2025-07-25 19:16:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 39/8 = ∑47/50, started new job
2025-07-25 19:16:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 39/8 = ∑47/50, started new job
2025-07-25 19:16:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 39/8 = ∑47/50, started new job
2025-07-25 19:16:57: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 39/8 = ∑47/50, started new job
2025-07-25 19:17:12: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 40/8 = ∑48/50, started new job
2025-07-25 19:21:29: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, starting new job
2025-07-25 19:21:30: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, starting new job
2025-07-25 19:23:32: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 48/2 = ∑50/50, started new job
2025-07-25 19:23:32: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 48/2 = ∑50/50, started new job
2025-07-25 19:24:53: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:25:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:26:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:26:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:27:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:28:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:28:55: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:29:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:30:16: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:30:56: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:31:35: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:32:15: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:32:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:33:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:34:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:34:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:35:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:36:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:36:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:37:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:38:13: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:38:53: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:39:33: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:40:14: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:40:54: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:41:34: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:42:14: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:42:52: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:43:31: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:44:10: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:44:49: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:45:28: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:46:08: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:46:48: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:47:29: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:48:09: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:48:49: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:49:28: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:50:07: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:50:47: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:51:26: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:52:05: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-25 19:52:37: BoTorchGenerator, failed: 182 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, job_failed
2025-07-25 19:53:39: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job
2025-07-25 19:54:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:54:46: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:55:26: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:56:05: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:56:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:57:25: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:58:05: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:58:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 19:59:26: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:00:06: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:00:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:01:24: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:02:03: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:02:43: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:03:23: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:04:02: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:04:42: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:05:22: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:06:02: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:06:43: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:07:22: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:08:03: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:08:43: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:09:22: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:10:01: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:10:40: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:11:20: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:11:59: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:12:40: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:13:18: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:13:58: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:14:38: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:15:18: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:15:58: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:16:39: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:17:19: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:17:59: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:18:37: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:19:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:19:56: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:20:37: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:21:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:21:56: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:22:37: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:23:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:23:57: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:24:36: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:25:16: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:25:56: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:26:36: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:27:15: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:27:54: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:28:33: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:29:13: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:29:52: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:30:32: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:31:12: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:31:50: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:32:30: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:33:09: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:33:49: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:34:28: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:35:09: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:35:49: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:36:29: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:37:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:37:48: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:38:26: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:39:06: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:39:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-25 20:40:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, new result: 98.4
2025-07-25 20:41:52: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-25 20:42:20: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:42:59: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:43:38: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:44:18: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:44:58: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:45:40: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:46:23: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:47:03: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:48:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:48:48: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:49:28: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:50:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:50:49: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:51:29: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:52:14: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:52:54: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:53:33: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:54:13: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:54:52: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:55:35: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:56:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:06: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 20:57:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-25 21:04:13: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 48 jobs, finished 16 jobs
2025-07-25 21:04:49: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 21:05:33: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 21:06:13: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-25 21:06:53: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:07:33: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:08:17: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:08:57: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:09:37: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:10:24: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:11:05: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:11:50: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:12:29: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:13:09: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:13:49: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:14:30: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:15:10: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:15:50: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:16:30: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-25 21:17:17: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 3/13 = ∑16/50, job_failed
2025-07-25 21:17:17: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 3/13 = ∑16/50, job_failed
2025-07-25 21:17:17: BoTorchGenerator, failed: 199 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 3/13 = ∑16/50, job_failed
2025-07-25 21:19:09: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 16 jobs, finished 3 jobs
2025-07-25 21:19:41: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:20:21: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:20:59: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:21:40: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:22:18: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:22:57: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:23:37: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:24:17: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:24:56: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:25:34: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:26:12: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:26:50: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 13 = ∑13/50, waiting for 13 jobs
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:38: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:39: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:27:40: BoTorchGenerator, failed: 202 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 13 = ∑13/50, job_failed
2025-07-25 21:32:37: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, waiting for 13 jobs, finished 13 jobs
2025-07-25 21:51:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #1/50
2025-07-25 21:52:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #2/50
2025-07-25 21:53:23: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #3/50
2025-07-25 21:54:20: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #4/50
2025-07-25 21:55:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #5/50
2025-07-25 21:56:15: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #6/50
2025-07-25 21:57:14: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #7/50
2025-07-25 21:58:13: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #8/50
2025-07-25 21:58:39: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #8/50
2025-07-25 21:59:37: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-25 22:00:04: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-25 22:01:04: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50
2025-07-25 22:02:03: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #11/50
2025-07-25 22:03:02: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #12/50
2025-07-25 22:04:00: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #13/50
2025-07-25 22:04:58: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #14/50
2025-07-25 22:05:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #15/50
2025-07-25 22:07:00: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #16/50
2025-07-25 22:07:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #17/50
2025-07-25 22:08:58: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50
2025-07-25 22:09:57: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #19/50
2025-07-25 22:10:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #20/50
2025-07-25 22:11:54: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #21/50
2025-07-25 22:12:52: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #22/50
2025-07-25 22:13:51: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #23/50
2025-07-25 22:14:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #24/50
2025-07-25 22:15:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #25/50
2025-07-25 22:16:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #26/50
2025-07-25 22:17:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #27/50
2025-07-25 22:18:48: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #28/50
2025-07-25 22:19:48: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #29/50
2025-07-25 22:20:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #30/50
2025-07-25 22:21:49: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #31/50
2025-07-25 22:22:47: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #32/50
2025-07-25 22:23:46: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #33/50
2025-07-25 22:24:44: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #34/50
2025-07-25 22:25:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #34/50
2025-07-25 22:26:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50
2025-07-25 22:26:37: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50
2025-07-25 22:27:35: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #36/50
2025-07-25 22:28:34: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #37/50
2025-07-25 22:29:32: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #38/50
2025-07-25 22:30:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #39/50
2025-07-25 22:31:29: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #40/50
2025-07-25 22:32:28: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #41/50
2025-07-25 22:33:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #42/50
2025-07-25 22:34:25: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #43/50
2025-07-25 22:35:24: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #44/50
2025-07-25 22:36:23: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #45/50
2025-07-25 22:37:22: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #46/50
2025-07-25 22:39:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #47/50
2025-07-25 22:40:48: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #48/50
2025-07-25 22:41:47: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #49/50
2025-07-25 22:42:14: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #49/50
2025-07-25 22:44:36: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #50/50
2025-07-25 22:45:37: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, requested 50 jobs, got 50, 86.82 s/job
2025-07-25 22:46:04: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #1/50 start
2025-07-25 22:46:41: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #2/50 start
2025-07-25 22:47:15: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #3/50 start
2025-07-25 22:47:48: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #4/50 start
2025-07-25 22:48:21: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #5/50 start
2025-07-25 22:48:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #6/50 start
2025-07-25 22:49:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #7/50 start
2025-07-25 22:50:00: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #8/50 start
2025-07-25 22:50:33: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #9/50 start
2025-07-25 22:51:06: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #10/50 start
2025-07-25 22:51:39: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #11/50 start
2025-07-25 22:52:13: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #12/50 start
2025-07-25 22:52:49: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #13/50 start
2025-07-25 22:53:23: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #14/50 start
2025-07-25 22:53:57: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #15/50 start
2025-07-25 22:54:29: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #16/50 start
2025-07-25 22:55:02: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #17/50 start
2025-07-25 22:55:36: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #18/50 start
2025-07-25 22:56:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #19/50 start
2025-07-25 22:56:44: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #20/50 start
2025-07-25 22:57:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #21/50 start
2025-07-25 22:57:50: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #22/50 start
2025-07-25 22:58:25: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #23/50 start
2025-07-25 22:58:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #24/50 start
2025-07-25 22:59:35: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #25/50 start
2025-07-25 23:00:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #26/50 start
2025-07-25 23:00:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #27/50 start
2025-07-25 23:01:19: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #28/50 start
2025-07-25 23:01:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #29/50 start
2025-07-25 23:02:26: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #30/50 start
2025-07-25 23:02:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #31/50 start
2025-07-25 23:03:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #32/50 start
2025-07-25 23:04:04: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #33/50 start
2025-07-25 23:04:37: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #34/50 start
2025-07-25 23:05:10: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #35/50 start
2025-07-25 23:05:42: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #36/50 start
2025-07-25 23:06:15: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #37/50 start
2025-07-25 23:06:48: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #38/50 start
2025-07-25 23:07:20: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #39/50 start
2025-07-25 23:07:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #40/50 start
2025-07-25 23:08:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #41/50 start
2025-07-25 23:08:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #42/50 start
2025-07-25 23:09:33: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #43/50 start
2025-07-25 23:10:06: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #44/50 start
2025-07-25 23:10:38: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #45/50 start
2025-07-25 23:11:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #46/50 start
2025-07-25 23:11:44: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #47/50 start
2025-07-25 23:12:16: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #48/50 start
2025-07-25 23:12:49: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #49/50 start
2025-07-25 23:13:21: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #50/50 start
2025-07-25 23:13:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:13:57: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-25 23:19:15: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 2 = ∑2/50, started new job
2025-07-25 23:19:15: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 2 = ∑2/50, started new job
2025-07-25 23:19:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/5 = ∑10/50, started new job
2025-07-25 23:19:28: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/6 = ∑11/50, started new job
2025-07-25 23:19:29: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/6 = ∑11/50, started new job
2025-07-25 23:19:29: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/6 = ∑11/50, started new job
2025-07-25 23:19:29: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/8 = ∑13/50, started new job
2025-07-25 23:19:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/11 = ∑16/50, started new job
2025-07-25 23:19:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/11 = ∑16/50, started new job
2025-07-25 23:19:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/11 = ∑16/50, started new job
2025-07-25 23:19:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/11 = ∑16/50, started new job
2025-07-25 23:19:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 2/3/11 = ∑16/50, started new job
2025-07-25 23:19:32: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 2/14 = ∑16/50, started new job
2025-07-25 23:19:33: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 2/14 = ∑16/50, started new job
2025-07-25 23:19:33: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 2/14 = ∑16/50, started new job
2025-07-25 23:19:33: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 2/14 = ∑16/50, started new job
2025-07-25 23:24:28: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:24:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:21: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:21: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:22: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:22: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:23: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:24: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:24: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:24: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:25: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-25 23:26:39: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, started new job
2025-07-25 23:26:39: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, started new job
2025-07-25 23:26:42: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, starting new job
2025-07-25 23:26:52: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, starting new job
2025-07-25 23:26:52: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, starting new job
2025-07-25 23:26:59: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, starting new job
2025-07-25 23:27:01: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 16/2 = ∑18/50, starting new job
2025-07-25 23:32:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 18/2 = ∑20/50, started new job
2025-07-25 23:32:11: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 18/2 = ∑20/50, started new job
2025-07-25 23:32:16: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:16: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:18: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/2/6 = ∑26/50, started new job
2025-07-25 23:32:32: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/8/1 = ∑27/50, started new job
2025-07-25 23:32:44: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), running/pending/unknown 28/1/3 = ∑32/50, started new job
2025-07-25 23:32:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best , running/pending/unknown 28/1/3 = ∑32/50, started new job
2025-07-25 23:32:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 28/1/3 = ∑32/50, started new job
2025-07-25 23:32:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), running/pending/unknown 28/1/3 = ∑32/50, started new job
2025-07-25 23:32:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 28/1/3 = ∑32/50, started new job
2025-07-25 23:33:36: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 23:33:38: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-25 23:38:45: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 30/2/2 = ∑34/50, started new job
2025-07-25 23:38:49: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 30/2/2 = ∑34/50, started new job
2025-07-25 23:38:52: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:38:52: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:38:54: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:51: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:56: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:39:57: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:44:17: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:44:21: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 32/2 = ∑34/50, starting new job
2025-07-25 23:44:27: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/unknown 32/2/2 = ∑36/50, started new job
2025-07-25 23:44:28: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/unknown 32/2/2 = ∑36/50, started new job
2025-07-25 23:44:43: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending/unknown 32/2/2/1 = ∑37/50, started new job
2025-07-25 23:45:19: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 35/2 = ∑37/50, starting new job
2025-07-25 23:45:20: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 35/2 = ∑37/50, starting new job
2025-07-25 23:45:42: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 35/2 = ∑37/50, starting new job
2025-07-25 23:45:42: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 35/2 = ∑37/50, starting new job
2025-07-25 23:45:42: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 35/2 = ∑37/50, starting new job
2025-07-25 23:45:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/unknown 35/2/5 = ∑42/50, started new job
2025-07-25 23:45:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 35/2/5 = ∑42/50, started new job
2025-07-25 23:45:53: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 35/2/5 = ∑42/50, started new job
2025-07-25 23:45:54: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending/unknown 35/2/5/1 = ∑43/50, started new job
2025-07-25 23:45:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending/unknown 35/2/5/1 = ∑43/50, started new job
2025-07-25 23:45:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending/unknown 35/2/5/1 = ∑43/50, started new job
2025-07-25 23:50:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best , running/out_of_memory/pending 41/2/2 = ∑45/50, started new job
2025-07-25 23:50:31: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 41/2/2 = ∑45/50, started new job
2025-07-25 23:50:55: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/unknown 43/2/2 = ∑47/50, started new job
2025-07-25 23:50:58: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending/unknown 43/2/1/1 = ∑47/50, started new job
2025-07-25 23:51:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/unknown 45/2/2 = ∑49/50, started new job
2025-07-25 23:51:30: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 45/2/2 = ∑49/50, started new job
2025-07-25 23:51:41: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory/pending 47/2/1 = ∑50/50, started new job
2025-07-25 23:55:09: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 48/2 = ∑50/50, job_failed
2025-07-25 23:55:09: BoTorchGenerator, failed: 215 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/out_of_memory 48/2 = ∑50/50, job_failed
2025-07-25 23:56:35: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, finishing jobs, finished 2 jobs
2025-07-25 23:57:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 23:57:58: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 23:58:41: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-25 23:59:24: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:00:08: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:00:51: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:01:36: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:02:20: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:03:04: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:03:47: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:04:30: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:05:13: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:05:58: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:06:51: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:07:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:08:21: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:09:34: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 00:10:10: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, new result: 98.15
2025-07-26 00:11:54: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-26 00:12:24: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:13:07: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:13:50: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:14:34: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:15:17: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:16:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:16:43: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:17:26: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:18:10: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:18:51: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:19:32: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:20:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:20:57: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:21:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:22:19: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:23:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:23:42: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:24:23: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:25:04: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:25:46: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:26:29: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:27:11: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:27:55: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:28:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:29:21: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:30:05: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:30:47: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:31:31: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:32:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:32:56: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:33:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:34:28: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:35:11: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:36:04: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:36:53: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:37:39: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:38:23: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:39:12: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:39:54: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:40:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:41:22: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:42:08: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:43:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:43:46: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:44:29: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:45:11: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:45:54: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:46:51: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:47:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:48:22: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:49:05: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:49:47: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:50:30: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:51:12: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:51:55: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:52:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:53:20: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:54:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:54:44: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:55:28: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:56:10: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:56:53: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:57:35: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:58:18: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:59:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 00:59:44: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:00:27: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:01:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:01:57: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:02:40: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:03:23: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:04:07: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:04:49: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:05:32: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:06:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:06:57: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:07:45: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:08:27: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:09:14: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:09:57: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:10:41: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:11:23: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:12:06: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:12:49: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:13:34: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:14:20: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:15:05: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:15:48: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:16:31: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:17:15: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:18:00: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:18:44: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:19:28: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:20:12: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:20:56: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:21:41: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:22:24: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:23:07: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:23:50: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:24:33: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:25:19: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:26:01: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:26:44: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 47 = ∑47/50, waiting for 47 jobs
2025-07-26 01:27:36: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:36: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:37: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:27:38: BoTorchGenerator, failed: 217 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/31 = ∑47/50, job_failed
2025-07-26 01:34:37: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 47 jobs, finished 16 jobs
2025-07-26 01:35:08: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 01:35:50: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 01:36:25: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, new result: 9.8
2025-07-26 01:38:08: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 13/17 = ∑30/50, waiting for 31 jobs, finished 1 job
2025-07-26 01:38:39: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:39:25: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:40:15: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:40:59: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:41:44: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:42:25: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:43:07: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:43:49: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:44:30: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:45:13: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:45:55: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:46:37: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:47:20: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 17 = ∑17/50, waiting for 17 jobs
2025-07-26 01:48:03: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 6 = ∑6/50, waiting for 6 jobs
2025-07-26 01:48:45: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 6 = ∑6/50, waiting for 6 jobs
2025-07-26 01:49:28: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 6 = ∑6/50, waiting for 6 jobs
2025-07-26 01:50:12: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 6 = ∑6/50, waiting for 6 jobs
2025-07-26 01:50:48: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 6 = ∑6/50, new result: 98.24
2025-07-26 01:52:32: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 6 jobs, finished 1 job
2025-07-26 01:53:07: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:53:58: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:54:40: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:55:21: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:56:07: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:56:48: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:57:28: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs
2025-07-26 01:58:19: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed
2025-07-26 01:58:19: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed
2025-07-26 01:58:19: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed
2025-07-26 01:58:20: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed
2025-07-26 01:58:20: BoTorchGenerator, failed: 233 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed
2025-07-26 02:00:46: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, waiting for 5 jobs, finished 5 jobs
2025-07-26 02:21:46: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #1/50
2025-07-26 02:22:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #1/50
2025-07-26 02:23:21: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #2/50
2025-07-26 02:24:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #3/50
2025-07-26 02:25:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #4/50
2025-07-26 02:26:38: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #5/50
2025-07-26 02:27:07: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #5/50
2025-07-26 02:28:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #6/50
2025-07-26 02:29:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #7/50
2025-07-26 02:30:25: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #8/50
2025-07-26 02:31:31: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-26 02:32:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-26 02:32:33: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50
2025-07-26 02:33:44: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50
2025-07-26 02:34:14: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50
2025-07-26 02:35:24: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #11/50
2025-07-26 02:36:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #12/50
2025-07-26 02:37:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #13/50
2025-07-26 02:38:45: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #14/50
2025-07-26 02:39:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #15/50
2025-07-26 02:41:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #16/50
2025-07-26 02:42:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #17/50
2025-07-26 02:42:46: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #17/50
2025-07-26 02:43:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50
2025-07-26 02:44:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #19/50
2025-07-26 02:46:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #20/50
2025-07-26 02:47:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #21/50
2025-07-26 02:48:17: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #22/50
2025-07-26 02:48:47: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #22/50
2025-07-26 02:49:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #23/50
2025-07-26 02:50:58: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #24/50
2025-07-26 02:52:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #25/50
2025-07-26 02:52:45: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #25/50
2025-07-26 02:53:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #26/50
2025-07-26 02:55:07: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #27/50
2025-07-26 02:56:11: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #28/50
2025-07-26 02:57:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #29/50
2025-07-26 02:58:21: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #30/50
2025-07-26 02:59:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #31/50
2025-07-26 03:00:32: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #32/50
2025-07-26 03:01:37: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #33/50
2025-07-26 03:02:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #34/50
2025-07-26 03:03:47: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50
2025-07-26 03:04:50: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #36/50
2025-07-26 03:05:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #36/50
2025-07-26 03:06:29: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #37/50
2025-07-26 03:07:36: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #38/50
2025-07-26 03:08:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #39/50
2025-07-26 03:09:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #40/50
2025-07-26 03:10:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #41/50
2025-07-26 03:15:53: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #42/50
2025-07-26 03:17:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #43/50
2025-07-26 03:18:17: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #44/50
2025-07-26 03:19:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #45/50
2025-07-26 03:20:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #46/50
2025-07-26 03:21:53: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #47/50
2025-07-26 03:22:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #47/50
2025-07-26 03:23:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #48/50
2025-07-26 03:24:22: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #48/50
2025-07-26 03:27:32: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #49/50
2025-07-26 03:28:38: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #50/50
2025-07-26 03:29:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, requested 50 jobs, got 50, 105.86 s/job
2025-07-26 03:30:21: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #1/50 start
2025-07-26 03:30:58: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #2/50 start
2025-07-26 03:31:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #3/50 start
2025-07-26 03:32:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #4/50 start
2025-07-26 03:32:53: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #5/50 start
2025-07-26 03:33:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #6/50 start
2025-07-26 03:34:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #7/50 start
2025-07-26 03:34:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #8/50 start
2025-07-26 03:35:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #9/50 start
2025-07-26 03:36:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #10/50 start
2025-07-26 03:36:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #11/50 start
2025-07-26 03:37:33: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #12/50 start
2025-07-26 03:38:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #13/50 start
2025-07-26 03:39:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #14/50 start
2025-07-26 03:39:41: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #15/50 start
2025-07-26 03:40:17: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #16/50 start
2025-07-26 03:41:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #17/50 start
2025-07-26 03:41:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #18/50 start
2025-07-26 03:42:18: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #19/50 start
2025-07-26 03:42:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #20/50 start
2025-07-26 03:43:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #21/50 start
2025-07-26 03:44:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #22/50 start
2025-07-26 03:44:40: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #23/50 start
2025-07-26 03:45:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #24/50 start
2025-07-26 03:45:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #25/50 start
2025-07-26 03:46:31: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #26/50 start
2025-07-26 03:47:06: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #27/50 start
2025-07-26 03:47:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #28/50 start
2025-07-26 03:48:19: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #29/50 start
2025-07-26 03:49:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #30/50 start
2025-07-26 03:49:50: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #31/50 start
2025-07-26 03:50:25: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #32/50 start
2025-07-26 03:51:01: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #33/50 start
2025-07-26 03:51:36: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #34/50 start
2025-07-26 03:52:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #35/50 start
2025-07-26 03:52:49: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #36/50 start
2025-07-26 03:53:32: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #37/50 start
2025-07-26 03:54:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #38/50 start
2025-07-26 03:54:44: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #39/50 start
2025-07-26 03:55:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #40/50 start
2025-07-26 03:55:58: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #41/50 start
2025-07-26 03:56:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #42/50 start
2025-07-26 03:57:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #43/50 start
2025-07-26 03:57:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #44/50 start
2025-07-26 03:58:24: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #45/50 start
2025-07-26 03:58:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #46/50 start
2025-07-26 03:59:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #47/50 start
2025-07-26 04:00:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #48/50 start
2025-07-26 04:00:45: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #49/50 start
2025-07-26 04:01:19: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #50/50 start
2025-07-26 04:02:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:02:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, starting new job
2025-07-26 04:08:01: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, unknown 1 = ∑1/50, started new job
2025-07-26 04:08:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, pending/unknown 1/1 = ∑2/50, started new job
2025-07-26 04:08:07: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, pending/unknown 1/7 = ∑8/50, started new job
2025-07-26 04:08:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 10/4 = ∑14/50, started new job
2025-07-26 04:08:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 10/4 = ∑14/50, started new job
2025-07-26 04:08:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 10/4 = ∑14/50, started new job
2025-07-26 04:08:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 10/4 = ∑14/50, started new job
2025-07-26 04:08:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 10/4 = ∑14/50, started new job
2025-07-26 04:08:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:08:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 10/4/2 = ∑16/50, started new job
2025-07-26 04:14:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:14:50: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:15:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:15:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:14: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:21: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:37: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:39: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, starting new job
2025-07-26 04:16:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 16/1 = ∑17/50, started new job
2025-07-26 04:19:27: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 17/1 = ∑18/50, started new job
2025-07-26 04:20:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 18/1 = ∑19/50, started new job
2025-07-26 04:20:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 18/1/1 = ∑20/50, started new job
2025-07-26 04:22:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 20/1 = ∑21/50, started new job
2025-07-26 04:22:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 21/1 = ∑22/50, started new job
2025-07-26 04:22:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 21/1/2 = ∑24/50, started new job
2025-07-26 04:22:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 21/1/2 = ∑24/50, started new job
2025-07-26 04:22:38: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 21/1/3 = ∑25/50, started new job
2025-07-26 04:22:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 21/4/1 = ∑26/50, started new job
2025-07-26 04:22:46: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 21/5/2 = ∑28/50, started new job
2025-07-26 04:22:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 24/4 = ∑28/50, started new job
2025-07-26 04:22:54: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 28/3 = ∑31/50, started new job
2025-07-26 04:22:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 28/3 = ∑31/50, started new job
2025-07-26 04:22:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 28/3 = ∑31/50, started new job
2025-07-26 04:23:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 28/4 = ∑32/50, started new job
2025-07-26 04:23:54: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-26 04:26:25: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-26 04:27:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-26 04:28:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, starting new job
2025-07-26 04:30:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, started new job
2025-07-26 04:30:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:30:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:30:47: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:30:49: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:30:56: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:31:00: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 32/1 = ∑33/50, starting new job
2025-07-26 04:31:24: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:31:24: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:31:25: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:31:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:31:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:31:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 33 = ∑33/50, starting new job
2025-07-26 04:32:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 33/1 = ∑34/50, started new job
2025-07-26 04:33:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 34/1 = ∑35/50, started new job
2025-07-26 04:33:33: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 35/1 = ∑36/50, started new job
2025-07-26 04:36:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 36/1 = ∑37/50, started new job
2025-07-26 04:37:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 37/1 = ∑38/50, started new job
2025-07-26 04:37:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 37/2 = ∑39/50, started new job
2025-07-26 04:37:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 37/2/1 = ∑40/50, started new job
2025-07-26 04:37:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 37/4 = ∑41/50, started new job
2025-07-26 04:37:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 37/5 = ∑42/50, started new job
2025-07-26 04:37:40: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending 42/1 = ∑43/50, started new job
2025-07-26 04:37:41: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 42/1/2 = ∑45/50, starting new job
2025-07-26 04:37:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 42/1/2 = ∑45/50, started new job
2025-07-26 04:37:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 42/1/2 = ∑45/50, started new job
2025-07-26 04:37:54: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 42/3/1 = ∑46/50, started new job
2025-07-26 04:38:06: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 46/1 = ∑47/50, started new job
2025-07-26 04:38:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/pending/unknown 46/1/1 = ∑48/50, started new job
2025-07-26 04:39:41: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, starting new job
2025-07-26 04:43:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), running/pending 48/1 = ∑49/50, started new job
2025-07-26 04:44:37: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running/unknown 49/1 = ∑50/50, started new job
2025-07-26 04:46:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:47:02: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:47:47: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:48:31: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:49:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:49:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:50:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:51:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:52:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:52:53: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:53:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:54:19: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, waiting for 50 jobs
2025-07-26 04:54:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 50 = ∑50/50, new result: 97.93
2025-07-26 04:56:39: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job
2025-07-26 04:57:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 04:57:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 04:58:34: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 04:59:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 04:59:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:00:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:01:27: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:02:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:02:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:03:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:04:18: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:05:00: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:05:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:06:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:07:07: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:07:50: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:08:33: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:09:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:09:58: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:10:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:11:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:12:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:12:53: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:13:36: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:14:18: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:15:01: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:15:44: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:16:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:17:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:17:56: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:18:42: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:19:25: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:20:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:20:51: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:21:35: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:22:17: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:23:00: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:23:43: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:24:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:25:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:26:03: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:26:45: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:27:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:28:13: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:28:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, waiting for 49 jobs
2025-07-26 05:29:32: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 49 = ∑49/50, new result: 9.16
2025-07-26 05:31:19: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-26 05:31:50: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:32:33: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:33:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:33:59: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:34:44: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:35:27: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:36:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:36:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:37:39: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:38:22: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:39:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:39:49: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:40:45: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:41:28: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:42:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:42:56: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:43:41: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:44:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:45:05: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:45:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:46:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:47:14: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:47:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:48:40: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:49:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:50:08: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:50:52: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:51:36: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:52:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:53:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:53:47: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:54:30: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:55:12: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:55:55: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:56:41: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:57:26: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:58:15: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:58:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 05:59:40: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:00:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:01:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:01:54: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:02:37: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:03:20: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:04:04: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:04:48: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:05:31: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:06:14: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:06:57: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:07:40: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:08:23: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:09:16: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 48 = ∑48/50, waiting for 48 jobs
2025-07-26 06:10:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:09: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:10:10: BoTorchGenerator, failed: 238 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 16/32 = ∑48/50, job_failed
2025-07-26 06:17:17: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 48 jobs, finished 16 jobs
2025-07-26 06:17:48: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-26 06:18:33: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-26 06:19:15: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 32 = ∑32/50, waiting for 32 jobs
2025-07-26 06:19:59: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:20:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:21:23: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:22:06: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:22:50: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:23:32: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:24:15: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:24:57: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:26:05: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:27:07: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:28:06: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:28:57: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:29:43: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 31 = ∑31/50, waiting for 31 jobs
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:41: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:30:42: BoTorchGenerator, failed: 254 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/16 = ∑31/50, job_failed
2025-07-26 06:37:50: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 31 jobs, finished 15 jobs
2025-07-26 06:38:25: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-26 06:39:11: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-26 06:40:03: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 16 = ∑16/50, waiting for 16 jobs
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:57: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:58: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:58: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:58: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:58: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:40:59: BoTorchGenerator, failed: 269 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 14/2 = ∑16/50, job_failed
2025-07-26 06:47:43: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 16 jobs, finished 14 jobs
2025-07-26 06:48:15: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 2 jobs
2025-07-26 06:48:58: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 2 jobs
2025-07-26 06:49:41: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 2 jobs
2025-07-26 06:50:23: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 2 = ∑2/50, waiting for 2 jobs
2025-07-26 06:51:12: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 2 = ∑2/50, job_failed
2025-07-26 06:51:12: BoTorchGenerator, failed: 283 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 2 = ∑2/50, job_failed
2025-07-26 06:52:37: BoTorchGenerator, failed: 285 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, waiting for 2 jobs, finished 2 jobs
submitit INFO (2025-07-24 13:50:06,079) - Starting with JobEnvironment(job_id=18601914, hostname=n1626, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 13:50:06,081) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/3/single_runs/18601914/18601914_submitted.pkl
slurmstepd: error: *** JOB 18601914 ON n1626 CANCELLED AT 2025-07-24T15:50:06 DUE TO TIME LIMIT ***