Experiment overview

Job Summary per Generation Node

Experiment parameters

Number of evaluations

Result names and types

Last progressbar status

Git-Version

Copy raw data to clipboard

Download »results.csv« as file

trial_index,submit_time,queue_time,start_time,end_time,run_time,program_string,exit_code,signal,hostname,OO_Info_SLURM_JOB_ID,arm_name,trial_status,generation_node,VAL_ACC,epochs,lr,batch_size,hidden_size,dropout,num_dense_layers,weight_decay,activation,init
0,,,,,,,,,,,0_0,FAILED,SOBOL,,76,0.055108410704135898927624737098,3296,1919,0.2823258936405181884765625,2,0.3202967345714569091796875,leaky_relu,normal
1,,,,,,,,,,,1_0,FAILED,SOBOL,,200,0.038496758051123472488264098956,1988,7750,0.124242893420159816741943359375,4,0.59415258280932903289794921875,leaky_relu,normal
2,,,,,,,,,,,2_0,FAILED,SOBOL,,116,0.080328420680761347671960947991,3049,2474,0.138288543093949556350708007812,3,0.2137086652219295501708984375,leaky_relu,normal
3,1753271083,24,1753271107,1753272870,1763,python3 .tests/mnist/train --epochs 39 --learning_rate 0.0137668233860284081 --batch_size 143 --hidden_size 4252 --dropout 0.45675678644329309464 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.9973303014412522316,0,,n1720,18592313,3_0,COMPLETED,SOBOL,8.83000000000000007105427357601,39,0.013766823386028408104930775835,143,4252,0.456756786443293094635009765625,1,0.997330301441252231597900390625,leaky_relu,normal
4,,,,,,,,,,,4_0,FAILED,SOBOL,,31,0.09244159757783637632044815291,1065,6114,0.376787260640412569046020507812,2,0.08037451095879077911376953125,leaky_relu,normal
5,,,,,,,,,,,5_0,FAILED,SOBOL,,149,0.000958351150434464337489470775,3907,3815,0.218715832568705081939697265625,4,0.865111893974244594573974609375,leaky_relu,normal
6,,,,,,,,,,,6_0,FAILED,SOBOL,,161,0.067904496547952303275863528143,802,6404,0.044269791338592767715454101562,3,0.452515085227787494659423828125,leaky_relu,normal
7,1753271083,24,1753271107,1753271557,450,python3 .tests/mnist/train --epochs 92 --learning_rate 0.02637119578570127493 --batch_size 2174 --hidden_size 70 --dropout 0.36275675054639577866 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.72748677432537078857,0,,n1720,18592311,7_0,COMPLETED,SOBOL,10.14000000000000056843418860808,92,0.026371195785701274927337323106,2174,70,0.362756750546395778656005859375,1,0.72748677432537078857421875,leaky_relu,normal
8,1753271083,23,1753271106,1753274216,3110,python3 .tests/mnist/train --epochs 101 --learning_rate 0.08183169904369860803 --batch_size 606 --hidden_size 3180 --dropout 0.07874552812427282333 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.0492831813171505928,0,,n1720,18592315,8_0,COMPLETED,SOBOL,8.35999999999999943156581139192,101,0.081831699043698608031505159488,606,3180,0.078745528124272823333740234375,1,0.049283181317150592803955078125,leaky_relu,normal
9,,,,,,,,,,,9_0,FAILED,SOBOL,,174,0.025032002712599931565362609831,2362,5470,0.264164520893245935440063476562,3,0.771381795406341552734375,leaky_relu,normal
10,,,,,,,,,,,10_0,FAILED,SOBOL,,139,0.056318195751030003526338418851,1396,706,0.496227749623358249664306640625,4,0.42124517448246479034423828125,leaky_relu,normal
11,1753271083,56,1753271139,1753274692,3553,python3 .tests/mnist/train --epochs 18 --learning_rate 0.04946855375478044492 --batch_size 3600 --hidden_size 7048 --dropout 0.15820572571828961372 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0.63357811328023672104,0,,n1720,18592321,11_0,COMPLETED,SOBOL,9.8000000000000007105427357601,18,0.049468553754780444919880721955,3600,7048,0.158205725718289613723754882812,2,0.633578113280236721038818359375,leaky_relu,normal
12,1753271083,24,1753271107,1753272935,1828,python3 .tests/mnist/train --epochs 57 --learning_rate 0.06923051304798573391 --batch_size 2734 --hidden_size 7363 --dropout 0.23478756193071603775 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.2888227514922618866,0,,n1720,18592309,12_0,COMPLETED,SOBOL,11.33999999999999985789145284798,57,0.06923051304798573390986149434,2734,7363,0.234787561930716037750244140625,1,0.2888227514922618865966796875,leaky_relu,normal
13,,,,,,,,,,,13_0,FAILED,SOBOL,,124,0.03744703007210046369346301276,466,1539,0.420225747395306825637817382812,3,0.500318297185003757476806640625,leaky_relu,normal
14,,,,,,,,,,,14_0,FAILED,SOBOL,,183,0.093865989182982598193483170235,3476,4640,0.340750410221517086029052734375,4,0.182542945258319377899169921875,leaky_relu,normal
15,,,,,,,,,,,15_0,FAILED,SOBOL,,68,0.01213245187224820144378245601,1784,2854,0.002739472780376672744750976562,2,0.90380430035293102264404296875,leaky_relu,normal
16,,,,,,,,,,,16_0,FAILED,SOBOL,,58,0.098083586611784998043894745479,2500,6328,0.442837075795978307723999023438,3,0.58405058085918426513671875,leaky_relu,normal
17,1753271165,33,1753271198,1753273162,1964,python3 .tests/mnist/train --epochs 182 --learning_rate 0.00849658845867961622 --batch_size 680 --hidden_size 402 --dropout 0.15168993175029754639 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.3608514862135052681,0,,n1720,18592328,17_0,COMPLETED,SOBOL,11.34999999999999964472863211995,182,0.008496588458679616220892150125,680,402,0.15168993175029754638671875,1,0.360851486213505268096923828125,leaky_relu,normal
18,,,,,,,,,,,18_0,FAILED,SOBOL,,122,0.072865810783673087835943249502,3834,5742,0.103479854296892881393432617188,2,0.948474711738526821136474609375,leaky_relu,normal
19,,,,,,,,,,,19_0,FAILED,SOBOL,,46,0.0332288550564087958871795081,1438,3931,0.30452372692525386810302734375,4,0.23113492317497730255126953125,leaky_relu,normal
20,,,,,,,,,,,20_0,FAILED,SOBOL,,13,0.060685754386149350081414155511,324,2070,0.349340818356722593307495117188,3,0.816067569889128208160400390625,leaky_relu,normal
21,1753271174,24,1753271198,1753274487,3289,python3 .tests/mnist/train --epochs 131 --learning_rate 0.04608019823227078177 --batch_size 2656 --hidden_size 4400 --dropout 0.05820499919354915619 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.09785988554358482361,0,,n1720,18592335,21_0,COMPLETED,SOBOL,15.38000000000000078159700933611,131,0.046080198232270781766306555483,2656,4400,0.05820499919354915618896484375,1,0.0978598855435848236083984375,leaky_relu,normal
22,,,,,,,,,,,22_0,FAILED,SOBOL,,168,0.085220620244089523298569588405,1546,1747,0.196502970065921545028686523438,2,0.71732558123767375946044921875,leaky_relu,normal
23,,,,,,,,,,,23_0,FAILED,SOBOL,,97,0.020665021664369852966158802587,3430,8178,0.39756627380847930908203125,4,0.493258695118129253387451171875,leaky_relu,normal
24,,,,,,,,,,,24_0,FAILED,SOBOL,,83,0.065101302229892465955884972573,1870,5035,0.176573611795902252197265625,4,0.78478169254958629608154296875,leaky_relu,normal
25,,,,,,,,,,,25_0,FAILED,SOBOL,,156,0.028202040625270454604622827333,3114,2714,0.479416736867278814315795898438,2,0.003965760581195354461669921875,leaky_relu,normal
26,,,,,,,,,,,26_0,FAILED,SOBOL,,145,0.090610949236899621106466895526,121,7543,0.27373595349490642547607421875,1,0.686219700612127780914306640625,leaky_relu,normal
27,1753271175,54,1753271229,1753273610,2381,python3 .tests/mnist/train --epochs 24 --learning_rate 0.00376173005811870085 --batch_size 2835 --hidden_size 1104 --dropout 0.06877784384414553642 --activation leaky_relu --num_dense_layers 3 --init normal --weight_decay 0.39954457804560661316,0,,n1720,18592337,27_0,COMPLETED,SOBOL,11.34999999999999964472863211995,24,0.003761730058118700845831705593,2835,1104,0.068777843844145536422729492188,3,0.3995445780456066131591796875,leaky_relu,normal
28,,,,,,,,,,,28_0,FAILED,SOBOL,,39,0.07766833393489942782839108304,4021,790,0.01953594200313091278076171875,4,0.552900842390954494476318359375,leaky_relu,normal
29,,,,,,,,,,,29_0,FAILED,SOBOL,,106,0.015838322482537477153829996723,1243,6708,0.322398021351546049118041992188,2,0.26731093414127826690673828125,leaky_relu,normal
30,,,,,,,,,,,30_0,FAILED,SOBOL,,189,0.053036714085564019838248128735,2192,3544,0.430178411304950714111328125,1,0.917015254497528076171875,leaky_relu,normal
31,,,,,,,,,,,31_0,FAILED,SOBOL,,74,0.041156660207360987180358335991,1012,5363,0.225232103373855352401733398438,3,0.137284670956432819366455078125,leaky_relu,normal
32,,,,,,,,,,,32_0,FAILED,SOBOL,,72,0.079577871731761853912878734718,1326,7980,0.367808841168880462646484375,3,0.958831506781280040740966796875,leaky_relu,normal
33,1753271240,19,1753271259,1753274394,3135,python3 .tests/mnist/train --epochs 193 --learning_rate 0.01462445869343355354 --batch_size 3658 --hidden_size 1565 --dropout 0.03760313382372260094 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.2365941312164068222,0,,n1720,18592352,33_0,COMPLETED,SOBOL,21.98999999999999843680598132778,193,0.014624458693433553535534485945,3658,1565,0.037603133823722600936889648438,1,0.23659413121640682220458984375,leaky_relu,normal
34,,,,,,,,,,,34_0,FAILED,SOBOL,,108,0.054262581782601779223362825633,536,4602,0.20716637186706066131591796875,2,0.5712671615183353424072265625,leaky_relu,normal
35,,,,,,,,,,,35_0,FAILED,SOBOL,,35,0.039259127807058397108264102826,2420,2256,0.384769051801413297653198242188,4,0.358795464970171451568603515625,leaky_relu,normal
36,,,,,,,,,,,36_0,FAILED,SOBOL,,27,0.06721605679737403804541173713,3542,3985,0.46135112456977367401123046875,3,0.68787195719778537750244140625,leaky_relu,normal
37,,,,,,,,,,,37_0,FAILED,SOBOL,,141,0.027196446778718380565820211814,1722,5812,0.131164133083075284957885742188,1,0.476516126655042171478271484375,leaky_relu,normal
38,1753271287,33,1753271320,1753273265,1945,python3 .tests/mnist/train --epochs 154 --learning_rate 0.09165331921670585857 --batch_size 2800 --hidden_size 344 --dropout 0.11418878287076950073 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0.84111092705279588699,0,,n1720,18592360,38_0,COMPLETED,SOBOL,9.82000000000000028421709430404,154,0.091653319216705858574201215561,2800,344,0.114188782870769500732421875,2,0.841110927052795886993408203125,leaky_relu,normal
39,,,,,,,,,,,39_0,FAILED,SOBOL,,87,0.001683750135265290855460240671,404,6254,0.291803026106208562850952148438,4,0.119989283382892608642578125,leaky_relu,normal
40,,,,,,,,,,,40_0,FAILED,SOBOL,,94,0.058705144373700030024920692995,2979,5177,0.008889229502528905868530273438,4,0.656598952598869800567626953125,leaky_relu,normal
41,,,,,,,,,,,41_0,FAILED,SOBOL,,169,0.047219939611107111798737889785,201,3342,0.335181415081024169921875,2,0.382603399455547332763671875,leaky_relu,normal
42,,,,,,,,,,,42_0,FAILED,SOBOL,,135,0.08411727167731151977303483136,3226,6890,0.411727077793329954147338867188,1,0.81002364493906497955322265625,leaky_relu,normal
43,1753271303,22,1753271325,1753271503,178,python3 .tests/mnist/train --epochs 11 --learning_rate 0.02268202597750350671 --batch_size 2046 --hidden_size 987 --dropout 0.24582014046609401703 --activation leaky_relu --num_dense_layers 3 --init normal --weight_decay 0.02626229729503393173,0,,n1720,18592365,43_0,COMPLETED,SOBOL,11.34999999999999964472863211995,11,0.022682025977503506714016978663,2046,987,0.24582014046609401702880859375,3,0.026262297295033931732177734375,leaky_relu,normal
44,,,,,,,,,,,44_0,FAILED,SOBOL,,49,0.096283800744265318760639615903,868,1178,0.165850854944437742233276367188,4,0.9276622124016284942626953125,leaky_relu,normal
45,,,,,,,,,,,45_0,FAILED,SOBOL,,120,0.009820201199874280945101290285,2112,7601,0.49215412698686122894287109375,2,0.143063590861856937408447265625,leaky_relu,normal
46,1753271291,29,1753271320,1753274127,2807,python3 .tests/mnist/train --epochs 178 --learning_rate 0.07155458378298208688 --batch_size 1131 --hidden_size 2644 --dropout 0.25520807644352316856 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.53979769628494977951,0,,n1720,18592362,46_0,COMPLETED,SOBOL,17.57000000000000028421709430404,178,0.07155458378298208688139681044,1131,2644,0.255208076443523168563842773438,1,0.539797696284949779510498046875,leaky_relu,normal
47,,,,,,,,,,,47_0,FAILED,SOBOL,,61,0.035041025175806136959799630404,3845,4982,0.0893203318119049072265625,3,0.26496476493775844573974609375,leaky_relu,normal
48,1753271309,42,1753271351,1753272072,721,python3 .tests/mnist/train --epochs 66 --learning_rate 0.07357749096341431594 --batch_size 63 --hidden_size 236 --dropout 0.40313717629760503769 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0.19164050370454788208,0,,n1720,18592371,48_0,COMPLETED,SOBOL,9.58000000000000007105427357601,66,0.073577490963414315938706522502,63,236,0.403137176297605037689208984375,2,0.191640503704547882080078125,leaky_relu,normal
49,,,,,,,,,,,49_0,FAILED,SOBOL,,186,0.032429905107617382209461709408,2905,6618,0.190355120692402124404907226562,4,0.972348195500671863555908203125,leaky_relu,normal
50,,,,,,,,,,,50_0,FAILED,SOBOL,,126,0.098895117311086508382622639601,1812,3646,0.047174328006803989410400390625,3,0.336978076957166194915771484375,leaky_relu,normal
51,1753276726,24,1753276750,1753278404,1654,python3 .tests/mnist/train --epochs 53 --learning_rate 0.00779747860906645704 --batch_size 3184 --hidden_size 5896 --dropout 0.3578413953073322773 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.62354495562613010406,0,,n1719,18594397,51_0,COMPLETED,SOBOL,11.34999999999999964472863211995,53,0.007797478609066457039267206142,3184,5896,0.357841395307332277297973632812,1,0.62354495562613010406494140625,leaky_relu,normal
52,1753276726,24,1753276750,1753278561,1811,python3 .tests/mnist/train --epochs 20 --learning_rate 0.08599670806080103591 --batch_size 2254 --hidden_size 4166 --dropout 0.30859925691038370132 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0.45463186595588922501,0,,n1719,18594391,52_0,COMPLETED,SOBOL,10.08999999999999985789145284798,20,0.085996708060801035911069334361,2254,4166,0.308599256910383701324462890625,2,0.454631865955889225006103515625,leaky_relu,normal
53,,,,,,,,,,,53_0,FAILED,SOBOL,,136,0.019829864609614014647753776899,946,2436,0.095836632419377565383911132812,4,0.7403309680521488189697265625,leaky_relu,normal
54,,,,,,,,,,,54_0,FAILED,SOBOL,,172,0.061557109242957090688985744009,4083,7841,0.141117035411298274993896484375,3,0.07485454343259334564208984375,leaky_relu,normal
55,1753276726,27,1753276753,1753278348,1595,python3 .tests/mnist/train --epochs 105 --learning_rate 0.04534031863799319051 --batch_size 1177 --hidden_size 1961 --dropout 0.4517954275943338871 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.8546943245455622673,0,,n1719,18594398,55_0,COMPLETED,SOBOL,0.340000000000000024424906541753,105,0.045340318637993190509227758866,1177,1961,0.451795427594333887100219726562,1,0.854694324545562267303466796875,leaky_relu,normal
56,1753276726,24,1753276750,1753278422,1672,python3 .tests/mnist/train --epochs 87 --learning_rate 0.08821561592826619724 --batch_size 3776 --hidden_size 3072 --dropout 0.23190066823735833168 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.4235305245965719223,0,,n1719,18594394,56_0,COMPLETED,SOBOL,9.82000000000000028421709430404,87,0.088215615928266197243168278419,3776,3072,0.231900668237358331680297851562,1,0.42353052459657192230224609375,leaky_relu,normal
57,,,,,,,,,,,57_0,FAILED,SOBOL,,164,0.006099519598204643233696486959,1508,4810,0.425128228031098842620849609375,3,0.646620425395667552947998046875,leaky_relu,normal
58,,,,,,,,,,,58_0,FAILED,SOBOL,,152,0.062799721656553453286164767633,2442,1326,0.314418137539178133010864257812,4,0.043565080501139163970947265625,leaky_relu,normal
59,,,,,,,,,,,59_0,FAILED,SOBOL,,29,0.030633572480268777410428171493,750,7197,0.031087310053408145904541015625,2,0.7607956714928150177001953125,leaky_relu,normal
60,1753276726,24,1753276750,1753278222,1472,python3 .tests/mnist/train --epochs 43 --learning_rate 0.05067606650711969224 --batch_size 1608 --hidden_size 7006 --dropout 0.07590416120365262032 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.16018511261790990829,0,,n1719,18594399,60_0,COMPLETED,SOBOL,9.81000000000000049737991503207,43,0.050676066507119692239324848515,1608,7006,0.075904161203652620315551757812,1,0.160185112617909908294677734375,leaky_relu,normal
61,,,,,,,,,,,61_0,FAILED,SOBOL,,113,0.043428514780197297051600457962,3364,616,0.269143522717058658599853515625,3,0.87850103713572025299072265625,leaky_relu,normal
62,,,,,,,,,,,62_0,FAILED,SOBOL,,197,0.075409050436504185266350930306,386,5509,0.469941502902656793594360351562,4,0.305825226008892059326171875,leaky_relu,normal
63,,,,,,,,,,,63_0,FAILED,SOBOL,,78,0.018211551555804908475444392479,2590,3266,0.186629629693925380706787109375,2,0.530000356025993824005126953125,leaky_relu,normal
64,1753276726,27,1753276753,1753278774,2021,python3 .tests/mnist/train --epochs 81 --learning_rate 0.09064309630151838604 --batch_size 513 --hidden_size 4746 --dropout 0.03354253107681870461 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.70599614549428224564,0,,n1719,18594396,64_0,COMPLETED,SOBOL,9.74000000000000021316282072803,81,0.090643096301518386037621155538,513,4746,0.033542531076818704605102539062,1,0.705996145494282245635986328125,leaky_relu,normal
65,,,,,,,,,,,65_0,FAILED,SOBOL,,196,0.002671953294612467149499179797,2718,3008,0.371564733795821666717529296875,3,0.48963524401187896728515625,leaky_relu,normal
66,,,,,,,,,,,66_0,FAILED,SOBOL,,112,0.06620354809695855324847002521,1735,7261,0.389501382131129503250122070312,4,0.82799160666763782501220703125,leaky_relu,normal
67,1753276898,37,1753276935,1753278888,1953,python3 .tests/mnist/train --epochs 44 --learning_rate 0.0281823626642115424 --batch_size 3493 --hidden_size 1390 --dropout 0.20408244896680116653 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0.10186526831239461899,0,,n1714,18594505,67_0,COMPLETED,SOBOL,10.27999999999999936051153781591,44,0.028182362664211542402359356174,3493,1390,0.204082448966801166534423828125,2,0.101865268312394618988037109375,leaky_relu,normal
68,1753276893,11,1753276904,1753277090,186,python3 .tests/mnist/train --epochs 29 --learning_rate 0.05329732653740793663 --batch_size 2315 --hidden_size 552 --dropout 0.12752707721665501595 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.94460600242018699646,0,,n1719,18594484,68_0,COMPLETED,SOBOL,10.27999999999999936051153781591,29,0.053297326537407936630774685227,2315,552,0.127527077216655015945434570312,1,0.9446060024201869964599609375,leaky_relu,normal
69,,,,,,,,,,,69_0,FAILED,SOBOL,,150,0.040197790237329902862661867857,620,6942,0.465537955053150653839111328125,3,0.219561063684523105621337890625,leaky_relu,normal
70,,,,,,,,,,,70_0,FAILED,SOBOL,,163,0.078614902271796016641758342303,3649,3330,0.295013415161520242691040039062,4,0.588300297968089580535888671875,leaky_relu,normal
71,,,,,,,,,,,71_0,FAILED,SOBOL,,90,0.015565408397559077000038563199,1379,5573,0.109575050882995128631591796875,2,0.37302093021571636199951171875,leaky_relu,normal
72,,,,,,,,,,,72_0,FAILED,SOBOL,,102,0.070931449776049704247782301536,3956,6554,0.331910531036555767059326171875,2,0.913559730164706707000732421875,leaky_relu,normal
73,1753276897,7,1753276904,1753278665,1761,python3 .tests/mnist/train --epochs 173 --learning_rate 0.03563585147066042347 --batch_size 1049 --hidden_size 172 --dropout 0.01344234729185700417 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0.12590941600501537323,0,,n1719,18594489,73_0,COMPLETED,SOBOL,10.27999999999999936051153781591,173,0.035635851470660423467862187863,1049,172,0.013442347291857004165649414062,4,0.12590941600501537322998046875,leaky_relu,normal
74,,,,,,,,,,,74_0,FAILED,SOBOL,,137,0.095664478364214308725443913772,2127,5960,0.249396574683487415313720703125,3,0.5569519512355327606201171875,leaky_relu,normal
75,1753276897,38,1753276935,1753277540,605,python3 .tests/mnist/train --epochs 20 --learning_rate 0.01041883763298392246 --batch_size 817 --hidden_size 3710 --dropout 0.40747974487021565437 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.27906719688326120377,0,,n1718,18594504,75_0,COMPLETED,SOBOL,13.03999999999999914734871708788,20,0.010418837632983922458773662356,817,3710,0.407479744870215654373168945312,1,0.279067196883261203765869140625,leaky_relu,normal
76,,,,,,,,,,,76_0,FAILED,SOBOL,,53,0.083545205729734159372235069441,1939,2372,0.487483364529907703399658203125,2,0.67460011132061481475830078125,leaky_relu,normal
77,,,,,,,,,,,77_0,FAILED,SOBOL,,128,0.023233405978139488184952554661,3313,4102,0.168996226508170366287231445312,4,0.395843646489083766937255859375,leaky_relu,normal
78,,,,,,,,,,,78_0,FAILED,SOBOL,,188,0.058129266799241310903489932116,190,2025,0.093442390672862529754638671875,3,0.796783213503658771514892578125,leaky_relu,normal
79,,,,,,,,,,,79_0,FAILED,SOBOL,,63,0.047767509473487741733510603126,3033,7904,0.251513759139925241470336914062,1,0.0082612931728363037109375,leaky_relu,normal
80,,,,,,,,,,,80_0,FAILED,SOBOL,,63,0.062496438577678059544950883719,1258,3406,0.193564971443265676498413085938,4,0.44052938930690288543701171875,leaky_relu,normal
81,,,,,,,,,,,81_0,FAILED,SOBOL,,178,0.044378779003303503447597222475,3974,5241,0.3985230214893817901611328125,2,0.723176787607371807098388671875,leaky_relu,normal
82,1753276987,14,1753277001,1753278312,1311,python3 .tests/mnist/train --epochs 117 --learning_rate 0.08693375161029398557 --batch_size 994 --hidden_size 924 --dropout 0.35420392779633402824 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.09200879279524087906,0,,n1719,18594553,82_0,COMPLETED,SOBOL,10.42999999999999971578290569596,117,0.086933751610293985567778918266,994,924,0.354203927796334028244018554688,1,0.092008792795240879058837890625,leaky_relu,normal
83,,,,,,,,,,,83_0,FAILED,SOBOL,,50,0.018866037701070308141293452309,2241,6826,0.05136062391102313995361328125,3,0.8687967620790004730224609375,leaky_relu,normal
84,,,,,,,,,,,84_0,FAILED,SOBOL,,11,0.099882463021855813489757736079,3100,7665,0.100568431895226240158081054688,4,0.209641671739518642425537109375,leaky_relu,normal
85,,,,,,,,,,,85_0,FAILED,SOBOL,,132,0.006783349539246410694892652771,1917,1241,0.30551491864025592803955078125,2,0.98558843322098255157470703125,leaky_relu,normal
86,1753277060,28,1753277088,1753280371,3283,python3 .tests/mnist/train --epochs 169 --learning_rate 0.07456712245941161943 --batch_size 2852 --hidden_size 4918 --dropout 0.4477344057522714138 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.32373763620853424072,0,,n1719,18594587,86_0,COMPLETED,SOBOL,10.71000000000000085265128291212,169,0.074567122459411619428415463062,2852,4918,0.447734405752271413803100585938,1,0.32373763620853424072265625,leaky_relu,normal
87,,,,,,,,,,,87_0,FAILED,SOBOL,,95,0.031418063311651353575193468259,72,2580,0.1448723860085010528564453125,3,0.605543960817158222198486328125,leaky_relu,normal
88,,,,,,,,,,,88_0,FAILED,SOBOL,,85,0.075955190473608671930527691529,2672,1630,0.4292498640716075897216796875,3,0.1783092916011810302734375,leaky_relu,normal
89,,,,,,,,,,,89_0,FAILED,SOBOL,,154,0.017637294350750745697276045121,275,8044,0.228205812629312276840209960938,1,0.891620163805782794952392578125,leaky_relu,normal
90,,,,,,,,,,,90_0,FAILED,SOBOL,,143,0.051226018171105537624132608698,3415,2193,0.02641601301729679107666015625,2,0.292705914936959743499755859375,leaky_relu,normal
91,,,,,,,,,,,91_0,FAILED,SOBOL,,25,0.042858067713398492748488877169,1593,4538,0.317563097458332777023315429688,4,0.51187633164227008819580078125,leaky_relu,normal
92,,,,,,,,,,,92_0,FAILED,SOBOL,,34,0.063399978459067640734758697363,662,5876,0.27271954156458377838134765625,3,0.409305014647543430328369140625,leaky_relu,normal
93,1753277083,35,1753277118,1753278815,1697,python3 .tests/mnist/train --epochs 111 --learning_rate 0.0300128202749416241 --batch_size 2549 --hidden_size 4049 --dropout 0.07165629742667078972 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.62958736345171928406,0,,n1719,18594606,93_0,COMPLETED,SOBOL,0.110000000000000000555111512313,111,0.030012820274941624104636517245,2549,4049,0.071656297426670789718627929688,1,0.6295873634517192840576171875,leaky_relu,normal
94,,,,,,,,,,,94_0,FAILED,SOBOL,,193,0.088812061103899039848919017004,1453,6191,0.1833582036197185516357421875,2,0.06059822253882884979248046875,leaky_relu,normal
95,1753277082,36,1753277118,1753278176,1058,python3 .tests/mnist/train --epochs 70 --learning_rate 0.00547495725462213232 --batch_size 3787 --hidden_size 280 --dropout 0.47449420159682631493 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0.77502113115042448044,0,,n1719,18594601,95_0,COMPLETED,SOBOL,9.74000000000000021316282072803,70,0.005474957254622132320209804135,3787,280,0.474494201596826314926147460938,4,0.775021131150424480438232421875,leaky_relu,normal
96,,,,,,,,,,,96_0,FAILED,SOBOL,,73,0.068526854236423975774705752428,2065,2778,0.1209719441831111907958984375,4,0.063341450877487659454345703125,leaky_relu,normal
97,,,,,,,,,,,97_0,FAILED,SOBOL,,192,0.025775597638264299837906889934,883,5099,0.286879060324281454086303710938,2,0.850886382162570953369140625,leaky_relu,normal
98,1753277152,28,1753277180,1753278044,864,python3 .tests/mnist/train --epochs 107 --learning_rate 0.09306166948108003656 --batch_size 3894 --hidden_size 1040 --dropout 0.46033328212797641754 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.46674054302275180817,0,,n1719,18594654,98_0,COMPLETED,SOBOL,11.60999999999999943156581139192,107,0.093061669481080036558928725299,3894,1040,0.46033328212797641754150390625,1,0.46674054302275180816650390625,leaky_relu,normal
99,,,,,,,,,,,99_0,FAILED,SOBOL,,37,0.00036046572914347050796066152,1114,7479,0.134041156154125928878784179688,3,0.744519918225705623626708984375,leaky_relu,normal
100,1753282968,44,1753283012,1753286331,3319,python3 .tests/mnist/train --epochs 195 --learning_rate 0.03058169169766884049 --batch_size 3481 --hidden_size 3652 --dropout 0.04306645334325824581 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.26442731290179205539,0,,n1720,18597127,100_0,COMPLETED,BOTORCH_MODULAR,13.82000000000000028421709430404,195,0.03058169169766884049055377659,3481,3652,0.043066453343258245811675521963,1,0.26442731290179205538848350443,leaky_relu,normal
101,1753282966,13,1753282979,1753284803,1824,python3 .tests/mnist/train --epochs 197 --learning_rate 0.00689278173695362026 --batch_size 2800 --hidden_size 1505 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.13908261090412643579,0,,n1720,18597118,101_0,COMPLETED,BOTORCH_MODULAR,30.53999999999999914734871708788,197,0.006892781736953620258523400821,2800,1505,0,1,0.139082610904126435791283711296,leaky_relu,normal
102,,,,,,,,,,,102_0,FAILED,BOTORCH_MODULAR,,195,0.019306566713463080980917041529,3639,4510,0,3,0.818684420138184654192059497291,leaky_relu,normal
103,,,,,,,,,,,103_0,FAILED,BOTORCH_MODULAR,,191,0.041291815481474862525868019247,3774,4115,0.053822659259306508683362579859,1,0.028208974889267574015505601892,leaky_relu,normal
104,1753282967,12,1753282979,1753284776,1797,python3 .tests/mnist/train --epochs 195 --learning_rate 0.02157559856240785778 --batch_size 2041 --hidden_size 839 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597125,104_0,COMPLETED,BOTORCH_MODULAR,79.540000000000006252776074688882,195,0.021575598562407857783362885584,2041,839,0.5,1,0,leaky_relu,normal
105,1753282967,44,1753283011,1753285612,2601,python3 .tests/mnist/train --epochs 195 --learning_rate 0.01819798338743101526 --batch_size 3996 --hidden_size 967 --dropout 0 --activation leaky_relu --num_dense_layers 3 --init normal --weight_decay 0,0,,n1720,18597130,105_0,COMPLETED,BOTORCH_MODULAR,9.8000000000000007105427357601,195,0.018197983387431015261936906313,3996,967,0,3,0,leaky_relu,normal
106,1753282967,44,1753283011,1753284595,1584,python3 .tests/mnist/train --epochs 194 --learning_rate 0.01965453871407999223 --batch_size 3984 --hidden_size 1291 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18597131,106_0,COMPLETED,BOTORCH_MODULAR,0.289999999999999980015985556747,194,0.019654538714079992228755955352,3984,1291,0,1,1,leaky_relu,normal
107,,,,,,,,,,,107_0,FAILED,BOTORCH_MODULAR,,195,0.007882971748167882583557819487,3859,5101,0.5,1,0.125429921353063578148834267267,leaky_relu,normal
108,,,,,,,,,,,108_0,FAILED,BOTORCH_MODULAR,,197,0.015263156900154197653796117606,3540,7933,0,1,0,leaky_relu,normal
109,1753282966,13,1753282979,1753285924,2945,python3 .tests/mnist/train --epochs 193 --learning_rate 0.03823668156947277602 --batch_size 3512 --hidden_size 2455 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.80550311863610268848,0,,n1720,18597121,109_0,COMPLETED,BOTORCH_MODULAR,0.349999999999999977795539507497,193,0.038236681569472776021090254517,3512,2455,0.5,1,0.805503118636102688476796629402,leaky_relu,normal
110,1753282967,16,1753282983,1753284915,1932,python3 .tests/mnist/train --epochs 195 --learning_rate 0.0001 --batch_size 4077 --hidden_size 1904 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597123,110_0,COMPLETED,BOTORCH_MODULAR,98.159999999999996589394868351519,195,0.000100000000000000004792173602,4077,1904,0,1,0,leaky_relu,normal
111,1753282966,13,1753282979,1753285531,2552,python3 .tests/mnist/train --epochs 193 --learning_rate 0.04277851466259617597 --batch_size 678 --hidden_size 2980 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.78391284977016240099,0,,n1720,18597117,111_0,COMPLETED,BOTORCH_MODULAR,0.200000000000000011102230246252,193,0.042778514662596175965880718195,678,2980,0,1,0.783912849770162400986350803578,leaky_relu,normal
112,,,,,,,,,,,112_0,FAILED,BOTORCH_MODULAR,,192,0.033359855180716199707369185035,3481,7285,0,4,0,leaky_relu,normal
113,1753282967,15,1753282982,1753284707,1725,python3 .tests/mnist/train --epochs 196 --learning_rate 0.0001 --batch_size 3603 --hidden_size 1248 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18597124,113_0,COMPLETED,BOTORCH_MODULAR,11.34999999999999964472863211995,196,0.000100000000000000004792173602,3603,1248,0,1,1,leaky_relu,normal
114,1753282966,18,1753282984,1753284222,1238,python3 .tests/mnist/train --epochs 190 --learning_rate 0.04769493828287918458 --batch_size 3624 --hidden_size 127 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.93112019575098914359,0,,n1720,18597119,114_0,COMPLETED,BOTORCH_MODULAR,0.19000000000000000222044604925,190,0.047694938282879184576756870229,3624,127,0,1,0.931120195750989143590459207189,leaky_relu,normal
115,,,,,,,,,,,115_0,FAILED,BOTORCH_MODULAR,,198,0.017129362333191535533449822992,828,7346,0.004040462723401947592316574287,1,1,leaky_relu,normal
116,,,,,,,,,,,116_0,FAILED,BOTORCH_MODULAR,,192,0.047720485165004197791738249634,4024,2173,0.39799473018392472756232791653,3,0.265178501328491744715165623347,leaky_relu,normal
117,1753283056,25,1753283081,1753284642,1561,python3 .tests/mnist/train --epochs 199 --learning_rate 0.02380703369448993259 --batch_size 4003 --hidden_size 1083 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.29295802679632421128,0,,n1720,18597158,117_0,COMPLETED,BOTORCH_MODULAR,2.649999999999999911182158029987,199,0.023807033694489932590387937239,4003,1083,0,1,0.292958026796324211282751548424,leaky_relu,normal
118,,,,,,,,,,,118_0,FAILED,BOTORCH_MODULAR,,197,0.004738552181556899764447443602,2076,1476,0.423794139090322319329828815171,3,0.384702380980970592982970401863,leaky_relu,normal
119,,,,,,,,,,,119_0,FAILED,BOTORCH_MODULAR,,186,0.084260340727330684806872795889,481,7370,0,4,0.03146722101592633324962378083,leaky_relu,normal
120,1753283123,18,1753283141,1753284379,1238,python3 .tests/mnist/train --epochs 196 --learning_rate 0.02593053254998393606 --batch_size 600 --hidden_size 115 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0.13188626543519155199,0,,n1720,18597168,120_0,COMPLETED,BOTORCH_MODULAR,39.1499999999999985789145284798,196,0.025930532549983936058124456281,600,115,0,1,0.131886265435191551986804370244,leaky_relu,normal
121,1753283129,12,1753283141,1753284637,1496,python3 .tests/mnist/train --epochs 196 --learning_rate 0.04583835219925547905 --batch_size 3647 --hidden_size 873 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597169,121_0,COMPLETED,BOTORCH_MODULAR,26.94000000000000127897692436818,196,0.045838352199255479046957617584,3647,873,0,1,0,leaky_relu,normal
122,1753283136,39,1753283175,1753284921,1746,python3 .tests/mnist/train --epochs 192 --learning_rate 0.02509713705923324531 --batch_size 4092 --hidden_size 1457 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597178,122_0,COMPLETED,BOTORCH_MODULAR,41.32000000000000028421709430404,192,0.025097137059233245309775028886,4092,1457,0,1,0,leaky_relu,normal
123,,,,,,,,,,,123_0,FAILED,BOTORCH_MODULAR,,196,0.009687538442743068647922122238,3669,4129,0.5,1,1,leaky_relu,normal
124,,,,,,,,,,,124_0,FAILED,BOTORCH_MODULAR,,197,0.000100000000000000004792173602,3288,4147,0,3,0,leaky_relu,normal
125,,,,,,,,,,,125_0,FAILED,BOTORCH_MODULAR,,188,0.059074596658321147646386606311,2899,6056,0.5,1,0,leaky_relu,normal
126,,,,,,,,,,,126_0,FAILED,BOTORCH_MODULAR,,192,0.022109864764421411492678615218,4043,7134,0.458637061632871545491241249692,1,0.913128220852284733233261704299,leaky_relu,normal
127,,,,,,,,,,,127_0,FAILED,BOTORCH_MODULAR,,192,0.047911406326952538092811550996,171,4974,0.5,1,0,leaky_relu,normal
128,,,,,,,,,,,128_0,FAILED,BOTORCH_MODULAR,,188,0.066755637338968568106345458091,3120,7075,0,1,0.362606864861066746641427016584,leaky_relu,normal
129,,,,,,,,,,,129_0,FAILED,BOTORCH_MODULAR,,193,0.008441804939139291308336510156,3907,1336,0.347911525716278924313940024149,4,0,leaky_relu,normal
130,,,,,,,,,,,130_0,FAILED,BOTORCH_MODULAR,,199,0.020143315525484470351891275186,1499,8119,0.5,2,0,leaky_relu,normal
131,,,,,,,,,,,131_0,FAILED,BOTORCH_MODULAR,,196,0.000100000000000000004792173602,3731,929,0,4,1,leaky_relu,normal
132,,,,,,,,,,,132_0,FAILED,BOTORCH_MODULAR,,196,0.029449403628583559200393438005,3276,1449,0.492970021887913945057846376585,4,0,leaky_relu,normal
133,,,,,,,,,,,133_0,FAILED,BOTORCH_MODULAR,,199,0.003874027546281501744457331782,3836,3313,0.5,1,0,leaky_relu,normal
134,1753283215,24,1753283239,1753284367,1128,python3 .tests/mnist/train --epochs 188 --learning_rate 0.07841139247447685701 --batch_size 535 --hidden_size 20 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18597219,134_0,COMPLETED,BOTORCH_MODULAR,6.65000000000000035527136788005,188,0.07841139247447685700631581085,535,20,0.5,1,1,leaky_relu,normal
135,1753283257,22,1753283279,1753285227,1948,python3 .tests/mnist/train --epochs 197 --learning_rate 0.02506979152895386781 --batch_size 891 --hidden_size 1212 --dropout 0.28533272275819637764 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18597221,135_0,COMPLETED,BOTORCH_MODULAR,0.080000000000000001665334536938,197,0.025069791528953867809637046093,891,1212,0.285332722758196377643713503858,1,1,leaky_relu,normal
136,,,,,,,,,,,136_0,FAILED,BOTORCH_MODULAR,,193,0.034392105549901665073608825196,3952,5968,0.5,1,0,leaky_relu,normal
137,,,,,,,,,,,137_0,FAILED,BOTORCH_MODULAR,,197,0.038903195779815061272621790067,3020,758,0,3,1,leaky_relu,normal
138,,,,,,,,,,,138_0,FAILED,BOTORCH_MODULAR,,186,0.070082474491428820018157352933,3661,3024,0,4,1,leaky_relu,normal
139,,,,,,,,,,,139_0,FAILED,BOTORCH_MODULAR,,184,0.100000000000000005551115123126,1140,5768,0,3,1,leaky_relu,normal
140,,,,,,,,,,,140_0,FAILED,BOTORCH_MODULAR,,194,0.006881495557791727363250444682,1573,6567,0,1,0,leaky_relu,normal
141,,,,,,,,,,,141_0,FAILED,BOTORCH_MODULAR,,200,0.006582056053974797045491573755,857,4119,0.5,2,0,leaky_relu,normal
142,,,,,,,,,,,142_0,FAILED,BOTORCH_MODULAR,,192,0.07090682456628630281336711505,34,5822,0.5,4,0.947232197523503405278688660474,leaky_relu,normal
143,,,,,,,,,,,143_0,FAILED,BOTORCH_MODULAR,,187,0.084057804423241122560028770749,2239,2288,0.5,4,0.63465517984622266123295730722,leaky_relu,normal
144,1753283300,45,1753283345,1753286669,3324,python3 .tests/mnist/train --epochs 190 --learning_rate 0.03928034250201566102 --batch_size 4079 --hidden_size 5311 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18597226,144_0,COMPLETED,BOTORCH_MODULAR,0.100000000000000005551115123126,190,0.039280342502015661021452785917,4079,5311,0,1,1,leaky_relu,normal
145,,,,,,,,,,,145_0,FAILED,BOTORCH_MODULAR,,193,0.037913100052421634666988126128,524,4947,0,2,0,leaky_relu,normal
146,1753283333,38,1753283371,1753284742,1371,python3 .tests/mnist/train --epochs 188 --learning_rate 0.08264331912541537928 --batch_size 3796 --hidden_size 395 --dropout 0.0790660387734289094 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597233,146_0,COMPLETED,BOTORCH_MODULAR,18.80999999999999872102307563182,188,0.082643319125415379278187799628,3796,395,0.079066038773428909403051534355,1,0,leaky_relu,normal
147,1753283333,38,1753283371,1753284623,1252,python3 .tests/mnist/train --epochs 198 --learning_rate 0.01687629103711906983 --batch_size 2546 --hidden_size 41 --dropout 0 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0,0,,n1720,18597234,147_0,COMPLETED,BOTORCH_MODULAR,17.71999999999999886313162278384,198,0.016876291037119069832206363913,2546,41,0,4,0,leaky_relu,normal
148,,,,,,,,,,,148_0,FAILED,BOTORCH_MODULAR,,195,0.020738852031805365960792641999,1410,1134,0,4,0.660353596871946280089105130173,leaky_relu,normal
149,1753283348,27,1753283375,1753285121,1746,python3 .tests/mnist/train --epochs 195 --learning_rate 0.0001 --batch_size 3667 --hidden_size 906 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18597236,149_0,COMPLETED,BOTORCH_MODULAR,97.909999999999996589394868351519,195,0.000100000000000000004792173602,3667,906,0.5,1,0,leaky_relu,normal
150,1753290160,20,1753290180,1753293247,3067,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598609,150_0,COMPLETED,BOTORCH_MODULAR,84.340000000000003410605131648481,200,0.000100000000000000004792173602,8,8,0.5,1,0,leaky_relu,normal
151,,,,,,,,,,,151_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,1,0,leaky_relu,normal
152,1753290159,21,1753290180,1753290248,68,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598607,152_0,COMPLETED,BOTORCH_MODULAR,29.73999999999999843680598132778,10,0.000100000000000000004792173602,4096,8,0.5,1,0,leaky_relu,normal
153,,,,,,,,,,,153_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,1,0,leaky_relu,normal
154,1753290160,54,1753290214,1753291439,1225,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598615,154_0,COMPLETED,BOTORCH_MODULAR,80.040000000000006252776074688882,200,0.000100000000000000004792173602,4096,8,0.5,1,0,leaky_relu,normal
155,1753290159,21,1753290180,1753290970,790,python3 .tests/mnist/train --epochs 51 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598608,155_0,COMPLETED,BOTORCH_MODULAR,83.57999999999999829469743417576,51,0.000100000000000000004792173602,8,8,0.5,1,0,leaky_relu,normal
156,1753290162,50,1753290212,1753290566,354,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 2978 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598620,156_0,COMPLETED,BOTORCH_MODULAR,95.35999999999999943156581139192,10,0.000100000000000000004792173602,2978,8192,0.5,1,0,leaky_relu,normal
157,,,,,,,,,,,157_0,FAILED,BOTORCH_MODULAR,,59,0.000100000000000000004792173602,8,8192,0.5,1,0,leaky_relu,normal
158,,,,,,,,,,,158_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4163,0.5,1,0,leaky_relu,normal
159,1753290158,22,1753290180,1753293112,2932,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 2288 --hidden_size 2669 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598606,159_0,COMPLETED,BOTORCH_MODULAR,98.439999999999997726263245567679,200,0.000100000000000000004792173602,2288,2669,0.5,1,0,leaky_relu,normal
160,,,,,,,,,,,160_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4356,0.5,1,0,leaky_relu,normal
161,,,,,,,,,,,161_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2140,8192,0.5,1,0,leaky_relu,normal
162,1753290161,19,1753290180,1753290704,524,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 997 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598614,162_0,COMPLETED,BOTORCH_MODULAR,97.64000000000000056843418860808,10,0.000100000000000000004792173602,8,997,0.5,1,0,leaky_relu,normal
163,1753290160,20,1753290180,1753290819,639,python3 .tests/mnist/train --epochs 106 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598610,163_0,COMPLETED,BOTORCH_MODULAR,73.069999999999993178789736703038,106,0.000100000000000000004792173602,4096,8,0.5,1,0,leaky_relu,normal
164,,,,,,,,,,,164_0,FAILED,BOTORCH_MODULAR,,125,0.000100000000000000004792173602,8,797,0.5,1,0,leaky_relu,normal
165,1753290162,50,1753290212,1753291430,1218,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 2066 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598618,165_0,COMPLETED,BOTORCH_MODULAR,84.310000000000002273736754432321,200,0.000100000000000000004792173602,2066,8,0.5,1,0,leaky_relu,normal
166,1753290414,23,1753290437,1753290511,74,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 2066 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598733,166_0,COMPLETED,BOTORCH_MODULAR,36.14000000000000056843418860808,10,0.000100000000000000004792173602,2066,8,0.5,1,0,leaky_relu,normal
167,1753290415,18,1753290433,1753293031,2598,python3 .tests/mnist/train --epochs 103 --learning_rate 0.0001 --batch_size 4096 --hidden_size 5590 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598734,167_0,COMPLETED,BOTORCH_MODULAR,98.200000000000002842170943040401,103,0.000100000000000000004792173602,4096,5590,0.5,1,0,leaky_relu,normal
168,,,,,,,,,,,168_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,2124,7743,0.5,4,0,leaky_relu,normal
169,1753290450,13,1753290463,1753291413,950,python3 .tests/mnist/train --epochs 56 --learning_rate 0.0001 --batch_size 2089 --hidden_size 3499 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598740,169_0,COMPLETED,BOTORCH_MODULAR,97.909999999999996589394868351519,56,0.000100000000000000004792173602,2089,3499,0.5,1,0,leaky_relu,normal
170,1753290451,12,1753290463,1753293196,2733,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0.20543379504205286357 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598741,170_0,COMPLETED,BOTORCH_MODULAR,90.040000000000006252776074688882,200,0.000100000000000000004792173602,8,8,0.20543379504205286356643966883,1,0,leaky_relu,normal
171,1753290454,9,1753290463,1753291207,744,python3 .tests/mnist/train --epochs 128 --learning_rate 0.0001 --batch_size 1464 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598743,171_0,COMPLETED,BOTORCH_MODULAR,84.519999999999996020960679743439,128,0.000100000000000000004792173602,1464,8,0.5,1,0,leaky_relu,normal
172,1753290451,12,1753290463,1753290711,248,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 4096 --hidden_size 5752 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598742,172_0,COMPLETED,BOTORCH_MODULAR,94.590000000000003410605131648481,10,0.000100000000000000004792173602,4096,5752,0.5,1,0,leaky_relu,normal
173,1753290460,35,1753290495,1753293227,2732,python3 .tests/mnist/train --epochs 89 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598749,173_0,COMPLETED,BOTORCH_MODULAR,98.159999999999996589394868351519,89,0.000100000000000000004792173602,4096,8192,0.5,1,0,leaky_relu,normal
174,,,,,,,,,,,174_0,FAILED,BOTORCH_MODULAR,,77,0.000100000000000000004792173602,8,3491,0.5,1,0,leaky_relu,normal
175,,,,,,,,,,,175_0,FAILED,BOTORCH_MODULAR,,132,0.000100000000000000004792173602,1677,8192,0.5,1,0,leaky_relu,normal
176,1753290461,34,1753290495,1753290557,62,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8 --dropout 0.06016612314603260459 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598750,176_0,COMPLETED,BOTORCH_MODULAR,35.869999999999997442046151263639,10,0.000100000000000000004792173602,4096,8,0.060166123146032604585009551101,1,0,leaky_relu,normal
177,1753290456,7,1753290463,1753293265,2802,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 6137 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598744,177_0,COMPLETED,BOTORCH_MODULAR,96.82999999999999829469743417576,10,0.000100000000000000004792173602,8,6137,0.5,1,0,leaky_relu,normal
178,,,,,,,,,,,178_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,137,8192,0.058987692656217075026781060387,1,0,leaky_relu,normal
179,,,,,,,,,,,179_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1556,4858,0.5,1,0,leaky_relu,normal
180,1753290504,23,1753290527,1753290720,193,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 1027 --hidden_size 4111 --dropout 0.06609878439452951393 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598751,180_0,COMPLETED,BOTORCH_MODULAR,96.57999999999999829469743417576,10,0.000100000000000000004792173602,1027,4111,0.066098784394529513930294228885,1,0,leaky_relu,normal
181,,,,,,,,,,,181_0,FAILED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,8,8192,0,4,1,leaky_relu,normal
182,1753290691,25,1753290716,1753291881,1165,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 1021 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598805,182_0,COMPLETED,BOTORCH_MODULAR,86.599999999999994315658113919199,200,0.000100000000000000004792173602,1021,8,0.5,1,0,leaky_relu,normal
183,1753290741,37,1753290778,1753291663,885,python3 .tests/mnist/train --epochs 23 --learning_rate 0.0001 --batch_size 1639 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598810,183_0,COMPLETED,BOTORCH_MODULAR,97.64000000000000056843418860808,23,0.000100000000000000004792173602,1639,8192,0.5,1,0,leaky_relu,normal
184,,,,,,,,,,,184_0,FAILED,BOTORCH_MODULAR,,125,0.000100000000000000004792173602,27,8038,0.5,1,0,leaky_relu,normal
185,,,,,,,,,,,185_0,FAILED,BOTORCH_MODULAR,,134,0.000100000000000000004792173602,41,3694,0.5,1,0,leaky_relu,normal
186,1753290756,22,1753290778,1753290952,174,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598812,186_0,COMPLETED,BOTORCH_MODULAR,92.090000000000003410605131648481,10,0.000100000000000000004792173602,8,8,0,1,0,leaky_relu,normal
187,1753290767,11,1753290778,1753290945,167,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598815,187_0,COMPLETED,BOTORCH_MODULAR,83.959999999999993747223925311118,10,0.000100000000000000004792173602,8,8,0.5,1,0,leaky_relu,normal
188,1753290771,7,1753290778,1753293417,2639,python3 .tests/mnist/train --epochs 140 --learning_rate 0.0001 --batch_size 4096 --hidden_size 4603 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598818,188_0,COMPLETED,BOTORCH_MODULAR,98.200000000000002842170943040401,140,0.000100000000000000004792173602,4096,4603,0.5,1,0,leaky_relu,normal
189,1753290769,9,1753290778,1753292705,1927,python3 .tests/mnist/train --epochs 128 --learning_rate 0.0001 --batch_size 1741 --hidden_size 3400 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598816,189_0,COMPLETED,BOTORCH_MODULAR,98.400000000000005684341886080801,128,0.000100000000000000004792173602,1741,3400,0.5,1,0,leaky_relu,normal
190,1753290771,7,1753290778,1753291857,1079,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 3054 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598817,190_0,COMPLETED,BOTORCH_MODULAR,77.340000000000003410605131648481,200,0.000100000000000000004792173602,3054,8,0.5,1,0,leaky_relu,normal
191,,,,,,,,,,,191_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
192,,,,,,,,,,,192_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5783,0.5,1,0,leaky_relu,normal
193,,,,,,,,,,,193_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,5210,0.251163862861768449885602194627,1,0,leaky_relu,normal
194,1753290775,39,1753290814,1753292122,1308,python3 .tests/mnist/train --epochs 145 --learning_rate 0.0001 --batch_size 4096 --hidden_size 1220 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598822,194_0,COMPLETED,BOTORCH_MODULAR,97.629999999999995452526491135359,145,0.000100000000000000004792173602,4096,1220,0.5,1,0,leaky_relu,normal
195,1753290804,38,1753290842,1753292942,2100,python3 .tests/mnist/train --epochs 134 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598847,195_0,COMPLETED,BOTORCH_MODULAR,86.57999999999999829469743417576,134,0.000100000000000000004792173602,8,8,0.5,1,0,leaky_relu,normal
196,,,,,,,,,,,196_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,2594,0.5,1,0,leaky_relu,normal
197,1753290875,30,1753290905,1753293376,2471,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 4096 --hidden_size 2286 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18598849,197_0,COMPLETED,BOTORCH_MODULAR,98.290000000000006252776074688882,200,0.000100000000000000004792173602,4096,2286,0.5,1,0,leaky_relu,normal
198,1753290993,38,1753291031,1753291247,216,python3 .tests/mnist/train --epochs 10 --learning_rate 0.10000000000000000555 --batch_size 8 --hidden_size 8 --dropout 0 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 1,0,,n1720,18598850,198_0,COMPLETED,BOTORCH_MODULAR,9.74000000000000021316282072803,10,0.100000000000000005551115123126,8,8,0,4,1,leaky_relu,normal
199,,,,,,,,,,,199_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2607,4412,0.5,1,0,leaky_relu,normal
200,,,,,,,,,,,200_0,FAILED,BOTORCH_MODULAR,,75,0.000100000000000000004792173602,8,4063,0,1,0,leaky_relu,normal
201,,,,,,,,,,,201_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5367,0.5,1,0,leaky_relu,normal
202,,,,,,,,,,,202_0,FAILED,BOTORCH_MODULAR,,104,0.000100000000000000004792173602,8,4213,0.5,1,0,leaky_relu,normal
203,,,,,,,,,,,203_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
204,,,,,,,,,,,204_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5584,0.5,2,0,leaky_relu,normal
205,,,,,,,,,,,205_0,FAILED,BOTORCH_MODULAR,,165,0.000100000000000000004792173602,2454,5904,0.5,1,0,leaky_relu,normal
206,,,,,,,,,,,206_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
207,,,,,,,,,,,207_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4615,0,1,0,leaky_relu,normal
208,,,,,,,,,,,208_0,FAILED,BOTORCH_MODULAR,,121,0.000100000000000000004792173602,8,5000,0,1,0,leaky_relu,normal
209,,,,,,,,,,,209_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
210,1753298652,28,1753298680,1753298985,305,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8192 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599276,210_0,COMPLETED,BOTORCH_MODULAR,95.150000000000005684341886080801,10,0.000100000000000000004792173602,4096,8192,0,1,0,leaky_relu,normal
211,,,,,,,,,,,151_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,1,0,leaky_relu,normal
212,,,,,,,,,,,212_0,FAILED,BOTORCH_MODULAR,,106,0.000100000000000000004792173602,8,3936,0.5,2,0,leaky_relu,normal
213,,,,,,,,,,,213_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0,4,0.541781621395578283362226557074,leaky_relu,normal
214,,,,,,,,,,,214_0,FAILED,BOTORCH_MODULAR,,86,0.000100000000000000004792173602,8,2956,0,1,0,leaky_relu,normal
215,,,,,,,,,,,215_0,FAILED,BOTORCH_MODULAR,,100,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
216,,,,,,,,,,,216_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
217,,,,,,,,,,,217_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,7801,0.5,1,0,leaky_relu,normal
218,,,,,,,,,,,218_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
219,,,,,,,,,,,219_0,FAILED,BOTORCH_MODULAR,,63,0.000100000000000000004792173602,2827,8192,0.5,3,0,leaky_relu,normal
220,,,,,,,,,,,220_0,FAILED,BOTORCH_MODULAR,,58,0.000100000000000000004792173602,8,4015,0.5,1,0,leaky_relu,normal
221,,,,,,,,,,,221_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,3,0,leaky_relu,normal
222,,,,,,,,,,,222_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
223,,,,,,,,,,,223_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
224,,,,,,,,,,,224_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0.536090834026830553860065720073,leaky_relu,normal
225,1753298926,17,1753298943,1753301921,2978,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 4458 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599303,225_0,COMPLETED,BOTORCH_MODULAR,97.090000000000003410605131648481,10,0.000100000000000000004792173602,8,4458,0.5,1,0,leaky_relu,normal
226,,,,,,,,,,,226_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4332,0.5,1,0,leaky_relu,normal
227,,,,,,,,,,,227_0,FAILED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,8,8192,0.5,4,0,leaky_relu,normal
228,,,,,,,,,,,228_0,FAILED,BOTORCH_MODULAR,,33,0.000100000000000000004792173602,8,3836,0,2,0,leaky_relu,normal
229,1753298920,23,1753298943,1753302376,3433,python3 .tests/mnist/train --epochs 123 --learning_rate 0.0001 --batch_size 3007 --hidden_size 5748 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599299,229_0,COMPLETED,BOTORCH_MODULAR,98.310000000000002273736754432321,123,0.000100000000000000004792173602,3007,5748,0.5,1,0,leaky_relu,normal
230,,,,,,,,,,,230_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,7905,0.5,4,0.277885004105439314958658769683,leaky_relu,normal
231,,,,,,,,,,,231_0,FAILED,BOTORCH_MODULAR,,141,0.000100000000000000004792173602,8,4516,0.5,1,0,leaky_relu,normal
232,,,,,,,,,,,232_0,FAILED,BOTORCH_MODULAR,,52,0.000100000000000000004792173602,8,7325,0,4,0,leaky_relu,normal
233,,,,,,,,,,,233_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4954,0.5,1,0,leaky_relu,normal
234,,,,,,,,,,,234_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,4,0.302337967745609281777774413058,leaky_relu,normal
235,,,,,,,,,,,235_0,FAILED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,4096,8192,0.5,4,0.071446556211442371897568648365,leaky_relu,normal
236,,,,,,,,,,,236_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4023,7820,0.5,4,0.322183156844136509011633506816,leaky_relu,normal
237,,,,,,,,,,,237_0,FAILED,BOTORCH_MODULAR,,195,0.000100000000000000004792173602,27,6047,0.5,2,0,leaky_relu,normal
238,1753299178,27,1753299205,1753302137,2932,python3 .tests/mnist/train --epochs 200 --learning_rate 0.10000000000000000555 --batch_size 8 --hidden_size 8 --dropout 0.5 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0,0,,n1720,18599318,238_0,COMPLETED,BOTORCH_MODULAR,6.780000000000000248689957516035,200,0.100000000000000005551115123126,8,8,0.5,4,0,leaky_relu,normal
239,,,,,,,,,,,239_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0,leaky_relu,normal
240,,,,,,,,,,,240_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2424,7317,0.5,1,0,leaky_relu,normal
241,,,,,,,,,,,241_0,FAILED,BOTORCH_MODULAR,,35,0.000100000000000000004792173602,4096,8192,0.5,2,0,leaky_relu,normal
242,,,,,,,,,,,242_0,FAILED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,1469,8192,0.5,4,0.270261845737524097899040498305,leaky_relu,normal
243,,,,,,,,,,,243_0,FAILED,BOTORCH_MODULAR,,91,0.000100000000000000004792173602,8,5496,0.5,1,0,leaky_relu,normal
244,,,,,,,,,,,244_0,FAILED,BOTORCH_MODULAR,,158,0.000100000000000000004792173602,8,6177,0,4,0,leaky_relu,normal
245,,,,,,,,,,,245_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,0,leaky_relu,normal
246,,,,,,,,,,,246_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,1,leaky_relu,normal
247,,,,,,,,,,,247_0,FAILED,BOTORCH_MODULAR,,161,0.000100000000000000004792173602,4096,5255,0.5,1,0,leaky_relu,normal
248,,,,,,,,,,,248_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,223,7788,0.5,4,0.430191413087093244005387759898,leaky_relu,normal
249,,,,,,,,,,,191_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
250,,,,,,,,,,,250_0,FAILED,BOTORCH_MODULAR,,145,0.000100000000000000004792173602,4096,7681,0.5,2,0,leaky_relu,normal
251,,,,,,,,,,,251_0,FAILED,BOTORCH_MODULAR,,74,0.000100000000000000004792173602,8,4086,0,1,0,leaky_relu,normal
252,,,,,,,,,,,252_0,FAILED,BOTORCH_MODULAR,,71,0.000100000000000000004792173602,8,3891,0,1,0,leaky_relu,normal
253,,,,,,,,,,,253_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4853,0.5,1,0,leaky_relu,normal
254,,,,,,,,,,,254_0,FAILED,BOTORCH_MODULAR,,113,0.000100000000000000004792173602,8,3995,0.5,1,0,leaky_relu,normal
255,,,,,,,,,,,203_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
256,,,,,,,,,,,256_0,FAILED,BOTORCH_MODULAR,,72,0.000100000000000000004792173602,8,4628,0,2,0,leaky_relu,normal
257,,,,,,,,,,,221_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,3,0,leaky_relu,normal
258,1753307436,56,1753307492,1753310875,3383,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 4712 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599521,258_0,COMPLETED,BOTORCH_MODULAR,97.629999999999995452526491135359,10,0.000100000000000000004792173602,8,4712,0,1,0,leaky_relu,normal
259,,,,,,,,,,,259_0,FAILED,BOTORCH_MODULAR,,132,0.000100000000000000004792173602,8,6310,0,1,0,leaky_relu,normal
260,,,,,,,,,,,209_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
261,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
262,,,,,,,,,,,262_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,6395,0.5,1,0,leaky_relu,normal
263,,,,,,,,,,,263_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2420,5032,0.5,1,0,leaky_relu,normal
264,,,,,,,,,,,264_0,FAILED,BOTORCH_MODULAR,,95,0.000100000000000000004792173602,8,4894,0,1,0,leaky_relu,normal
265,,,,,,,,,,,206_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
266,1753307435,58,1753307493,1753309941,2448,python3 .tests/mnist/train --epochs 94 --learning_rate 0.0001 --batch_size 2374 --hidden_size 7063 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599519,266_0,COMPLETED,BOTORCH_MODULAR,98.269999999999996020960679743439,94,0.000100000000000000004792173602,2374,7063,0,1,0,leaky_relu,normal
267,,,,,,,,,,,267_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,0.59268357844235941556121360918,leaky_relu,normal
268,,,,,,,,,,,268_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,1047,8192,0.5,3,0,leaky_relu,normal
269,,,,,,,,,,,269_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,7521,0.5,4,0.518374063080345059972842136631,leaky_relu,normal
270,,,,,,,,,,,270_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,0.892345812246082559227033925708,leaky_relu,normal
271,,,,,,,,,,,271_0,FAILED,BOTORCH_MODULAR,,96,0.000100000000000000004792173602,8,2491,0,1,0,leaky_relu,normal
272,,,,,,,,,,,272_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,2,0,leaky_relu,normal
273,,,,,,,,,,,273_0,RUNNING,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
274,,,,,,,,,,,274_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,6239,0,4,0,leaky_relu,normal
275,,,,,,,,,,,191_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
276,1753307850,35,1753307885,1753307922,37,python3 .tests/mnist/train --epochs 200 --learning_rate 0.0001 --batch_size 903 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0.496617591470426889,247,,n1720,18599536,276_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,903,8192,0.5,4,0.496617591470426888999867287566,leaky_relu,normal
277,,,,,,,,,,,277_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,3166,0.5,4,0,leaky_relu,normal
278,,,,,,,,,,,278_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,6721,0,1,0,leaky_relu,normal
279,,,,,,,,,,,279_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,3855,8192,0.5,4,0.4364629599051870756376558802,leaky_relu,normal
280,1753307839,15,1753307854,1753310680,2826,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 3602 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599526,280_0,COMPLETED,BOTORCH_MODULAR,98.049999999999997157829056959599,10,0.000100000000000000004792173602,8,3602,0,1,0,leaky_relu,normal
281,,,,,,,,,,,281_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,577,2092,0.5,4,0.599760311982907690087074570329,leaky_relu,normal
282,,,,,,,,,,,246_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,1,leaky_relu,normal
283,,,,,,,,,,,283_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5137,0,2,0,leaky_relu,normal
284,1753307846,40,1753307886,1753308316,430,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 4096 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599527,284_0,COMPLETED,BOTORCH_MODULAR,95,10,0.000100000000000000004792173602,4096,8192,0.5,1,0,leaky_relu,normal
285,,,,,,,,,,,285_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0.660567933848947186348254945187,leaky_relu,normal
286,,,,,,,,,,,286_0,RUNNING,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,8,8192,0.5,4,0.833891358282042971517000751192,leaky_relu,normal
287,,,,,,,,,,,287_0,RUNNING,BOTORCH_MODULAR,,136,0.000100000000000000004792173602,8,3735,0.5,2,0,leaky_relu,normal
288,,,,,,,,,,,288_0,RUNNING,BOTORCH_MODULAR,,72,0.000100000000000000004792173602,4096,8103,0,4,0.276883511477188870575361079318,leaky_relu,normal
289,,,,,,,,,,,289_0,RUNNING,BOTORCH_MODULAR,,115,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
290,,,,,,,,,,,290_0,RUNNING,BOTORCH_MODULAR,,102,0.000100000000000000004792173602,2196,7256,0,2,0,leaky_relu,normal
291,,,,,,,,,,,291_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0,4,1,leaky_relu,normal
292,,,,,,,,,,,292_0,RUNNING,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,4096,8192,0,4,1,leaky_relu,normal
293,,,,,,,,,,,293_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4942,0.5,1,0,leaky_relu,normal
294,1753308294,25,1753308319,1753308554,235,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 8 --dropout 0 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 1,0,,n1720,18599548,294_0,COMPLETED,BOTORCH_MODULAR,11.34999999999999964472863211995,10,0.000100000000000000004792173602,8,8,0,4,1,leaky_relu,normal
295,,,,,,,,,,,295_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0,4,0.361213700145578686662162226639,leaky_relu,normal
296,,,,,,,,,,,296_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0.716756735736520766977264429443,leaky_relu,normal
297,,,,,,,,,,,297_0,FAILED,BOTORCH_MODULAR,,77,0.000100000000000000004792173602,2704,8192,0.5,2,0,leaky_relu,normal
298,,,,,,,,,,,298_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0.857130097730807838374289531203,leaky_relu,normal
299,1753308316,29,1753308345,1753308635,290,python3 .tests/mnist/train --epochs 10 --learning_rate 0.10000000000000000555 --batch_size 4096 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 1,0,,n1720,18599557,299_0,COMPLETED,BOTORCH_MODULAR,10.09999999999999964472863211995,10,0.100000000000000005551115123126,4096,8192,0.5,1,1,leaky_relu,normal
300,,,,,,,,,,,300_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,3,0,leaky_relu,normal
301,,,,,,,,,,,301_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2271,8192,0.5,1,0,leaky_relu,normal
302,,,,,,,,,,,302_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,2049,8192,0,4,1,leaky_relu,normal
303,,,,,,,,,,,303_0,FAILED,BOTORCH_MODULAR,,70,0.000100000000000000004792173602,8,4008,0,1,0,leaky_relu,normal
304,,,,,,,,,,,304_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4885,0.5,1,0,leaky_relu,normal
305,,,,,,,,,,,203_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
306,,,,,,,,,,,306_0,FAILED,BOTORCH_MODULAR,,121,0.000100000000000000004792173602,8,4075,0.5,1,0,leaky_relu,normal
307,,,,,,,,,,,221_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,3,0,leaky_relu,normal
308,,,,,,,,,,,308_0,FAILED,BOTORCH_MODULAR,,185,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
309,,,,,,,,,,,309_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,3216,8192,0,4,1,leaky_relu,normal
310,,,,,,,,,,,310_0,FAILED,BOTORCH_MODULAR,,74,0.000100000000000000004792173602,8,3988,0,1,0,leaky_relu,normal
311,,,,,,,,,,,311_0,FAILED,BOTORCH_MODULAR,,105,0.000100000000000000004792173602,8,3962,0.5,1,0,leaky_relu,normal
312,,,,,,,,,,,312_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5120,0.5,1,0,leaky_relu,normal
313,,,,,,,,,,,313_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
314,,,,,,,,,,,314_0,FAILED,BOTORCH_MODULAR,,96,0.000100000000000000004792173602,8,3099,0,1,0,leaky_relu,normal
315,,,,,,,,,,,315_0,FAILED,BOTORCH_MODULAR,,156,0.000100000000000000004792173602,8,5240,0.5,1,0,leaky_relu,normal
316,,,,,,,,,,,203_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,3,0,leaky_relu,normal
317,,,,,,,,,,,317_0,FAILED,BOTORCH_MODULAR,,130,0.000100000000000000004792173602,8,3384,0.5,2,0,leaky_relu,normal
318,,,,,,,,,,,318_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8050,0.5,1,0,leaky_relu,normal
319,,,,,,,,,,,319_0,FAILED,BOTORCH_MODULAR,,90,0.000100000000000000004792173602,8,6747,0.5,3,0,leaky_relu,normal
320,,,,,,,,,,,320_0,FAILED,BOTORCH_MODULAR,,11,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
321,,,,,,,,,,,321_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2313,4848,0.5,1,0,leaky_relu,normal
322,,,,,,,,,,,322_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8094,0.5,4,0.569605517828528751245187322638,leaky_relu,normal
323,,,,,,,,,,,209_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
324,,,,,,,,,,,324_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0.508826380193720506639465384069,leaky_relu,normal
325,,,,,,,,,,,216_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
326,,,,,,,,,,,326_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0.715371051846681371699787632679,leaky_relu,normal
327,,,,,,,,,,,327_0,FAILED,BOTORCH_MODULAR,,31,0.000100000000000000004792173602,8,5186,0,2,0,leaky_relu,normal
328,,,,,,,,,,,328_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,1076,8192,0.5,3,0,leaky_relu,normal
329,,,,,,,,,,,329_0,FAILED,BOTORCH_MODULAR,,152,0.000100000000000000004792173602,4096,5585,0.5,1,0,leaky_relu,normal
330,,,,,,,,,,,330_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,7162,0.5,1,0,leaky_relu,normal
331,,,,,,,,,,,331_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8,0.5,4,0.556943094591489495925884511962,leaky_relu,normal
332,,,,,,,,,,,332_0,FAILED,BOTORCH_MODULAR,,119,0.000100000000000000004792173602,8,5327,0,1,0,leaky_relu,normal
333,,,,,,,,,,,333_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4068,0.5,1,0,leaky_relu,normal
334,,,,,,,,,,,334_0,FAILED,BOTORCH_MODULAR,,48,0.000100000000000000004792173602,4096,8192,0.5,2,0,leaky_relu,normal
335,,,,,,,,,,,335_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,366,8040,0.5,4,0.288190186306194473520037035996,leaky_relu,normal
336,,,,,,,,,,,336_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5400,0.5,2,0,leaky_relu,normal
337,,,,,,,,,,,337_0,FAILED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,4096,8192,0.5,4,0,leaky_relu,normal
338,,,,,,,,,,,338_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0,4,1,leaky_relu,normal
339,,,,,,,,,,,339_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,3059,8192,0.5,4,1,leaky_relu,normal
340,,,,,,,,,,,340_0,FAILED,BOTORCH_MODULAR,,37,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
341,,,,,,,,,,,341_0,FAILED,BOTORCH_MODULAR,,152,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
342,,,,,,,,,,,342_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,1,leaky_relu,normal
343,,,,,,,,,,,343_0,FAILED,BOTORCH_MODULAR,,56,0.000100000000000000004792173602,2692,8192,0,4,0,leaky_relu,normal
344,,,,,,,,,,,344_0,FAILED,BOTORCH_MODULAR,,91,0.000100000000000000004792173602,8,2436,0,2,0,leaky_relu,normal
345,,,,,,,,,,,345_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4196,0.5,1,0,leaky_relu,normal
346,,,,,,,,,,,346_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,7502,0.5,4,0.629123087996016905343310554599,leaky_relu,normal
347,,,,,,,,,,,347_0,FAILED,BOTORCH_MODULAR,,106,0.000100000000000000004792173602,1795,8192,0.5,2,0,leaky_relu,normal
348,,,,,,,,,,,348_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,5020,0.5,4,0,leaky_relu,normal
349,,,,,,,,,,,349_0,FAILED,BOTORCH_MODULAR,,69,0.000100000000000000004792173602,8,4400,0,1,0,leaky_relu,normal
350,,,,,,,,,,,350_0,FAILED,BOTORCH_MODULAR,,22,0.000100000000000000004792173602,3986,8082,0.5,4,0.222218087117110663086805288913,leaky_relu,normal
351,1753317151,10,1753317161,1753319803,2642,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 2610 --hidden_size 8192 --dropout 0 --activation leaky_relu --num_dense_layers 2 --init normal --weight_decay 0,0,,n1720,18599813,351_0,COMPLETED,BOTORCH_MODULAR,87.709999999999993747223925311118,10,0.000100000000000000004792173602,2610,8192,0,2,0,leaky_relu,normal
352,,,,,,,,,,,352_0,FAILED,BOTORCH_MODULAR,,85,0.000100000000000000004792173602,8,3424,0.5,1,0,leaky_relu,normal
353,,,,,,,,,,,353_0,FAILED,BOTORCH_MODULAR,,179,0.100000000000000005551115123126,4096,8192,0,4,0,leaky_relu,normal
354,,,,,,,,,,,354_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,3214,4284,0.5,1,0,leaky_relu,normal
355,,,,,,,,,,,355_0,FAILED,BOTORCH_MODULAR,,133,0.000100000000000000004792173602,152,7945,0.5,1,0,leaky_relu,normal
356,,,,,,,,,,,356_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,7190,0.5,4,0.934654615220081086413017601444,leaky_relu,normal
357,,,,,,,,,,,357_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,407,3266,0,4,0.487755994766489509029128157636,leaky_relu,normal
358,,,,,,,,,,,291_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0,4,1,leaky_relu,normal
359,1753317163,26,1753317189,1753317465,276,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 1929 --hidden_size 7168 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599820,359_0,COMPLETED,BOTORCH_MODULAR,96.209999999999993747223925311118,10,0.000100000000000000004792173602,1929,7168,0,1,0,leaky_relu,normal
360,,,,,,,,,,,360_0,FAILED,BOTORCH_MODULAR,,194,0.000100000000000000004792173602,8,8192,0.5,2,0,leaky_relu,normal
361,1753317164,25,1753317189,1753319496,2307,python3 .tests/mnist/train --epochs 111 --learning_rate 0.0001 --batch_size 4096 --hidden_size 5691 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18599821,361_0,COMPLETED,BOTORCH_MODULAR,98.260000000000005115907697472721,111,0.000100000000000000004792173602,4096,5691,0,1,0,leaky_relu,normal
362,,,,,,,,,,,362_0,FAILED,BOTORCH_MODULAR,,114,0.000100000000000000004792173602,8,4055,0.5,1,0,leaky_relu,normal
363,,,,,,,,,,,363_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5092,0.5,1,0,leaky_relu,normal
364,,,,,,,,,,,313_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
365,,,,,,,,,,,365_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,1,leaky_relu,normal
366,,,,,,,,,,,366_0,RUNNING,BOTORCH_MODULAR,,94,0.000100000000000000004792173602,8,3774,0,1,0,leaky_relu,normal
367,,,,,,,,,,,367_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4872,0.5,1,0,leaky_relu,normal
368,,,,,,,,,,,368_0,RUNNING,BOTORCH_MODULAR,,109,0.000100000000000000004792173602,8,4045,0.5,1,0,leaky_relu,normal
369,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
370,,,,,,,,,,,370_0,RUNNING,BOTORCH_MODULAR,,85,0.000100000000000000004792173602,8,4650,0.5,2,0,leaky_relu,normal
371,,,,,,,,,,,371_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,7941,0.5,4,0.554931551092388453128023684258,leaky_relu,normal
372,,,,,,,,,,,372_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
373,,,,,,,,,,,373_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4046,8192,0.5,4,0.441960739682177916609617795984,leaky_relu,normal
374,,,,,,,,,,,374_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,5769,0.5,1,0,leaky_relu,normal
375,,,,,,,,,,,375_0,RUNNING,BOTORCH_MODULAR,,167,0.000100000000000000004792173602,2521,4732,0.5,1,0,leaky_relu,normal
376,,,,,,,,,,,337_0,ABANDONED,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,4096,8192,0.5,4,0,leaky_relu,normal
377,,,,,,,,,,,153_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,1,0,leaky_relu,normal
378,,,,,,,,,,,239_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0,leaky_relu,normal
379,,,,,,,,,,,379_0,RUNNING,BOTORCH_MODULAR,,71,0.000100000000000000004792173602,8,7168,0,4,0,leaky_relu,normal
380,,,,,,,,,,,380_0,RUNNING,BOTORCH_MODULAR,,87,0.000100000000000000004792173602,8,2997,0.5,1,0,leaky_relu,normal
381,,,,,,,,,,,381_0,RUNNING,BOTORCH_MODULAR,,36,0.000100000000000000004792173602,8,5662,0,3,0,leaky_relu,normal
382,,,,,,,,,,,382_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4450,0,4,0,leaky_relu,normal
383,,,,,,,,,,,383_0,RUNNING,BOTORCH_MODULAR,,88,0.000100000000000000004792173602,8,8065,0.5,1,0,leaky_relu,normal
384,1753326271,56,1753326327,1753326352,25,python3 .tests/mnist/train --epochs 200 --learning_rate 0.10000000000000000555 --batch_size 650 --hidden_size 8032 --dropout 0.5 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0.58356592911903870657,247,,n1720,18599983,384_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,650,8032,0.5,4,0.583565929119038706573974195635,leaky_relu,normal
385,,,,,,,,,,,385_0,RUNNING,BOTORCH_MODULAR,,166,0.000100000000000000004792173602,8,5272,0.5,4,0,leaky_relu,normal
386,,,,,,,,,,,206_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
387,,,,,,,,,,,191_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
388,,,,,,,,,,,313_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
389,1753326740,32,1753326772,1753329834,3062,python3 .tests/mnist/train --epochs 200 --learning_rate 0.10000000000000000555 --batch_size 142 --hidden_size 400 --dropout 0 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0.54879322743063208012,0,,n1720,18600042,389_0,COMPLETED,BOTORCH_MODULAR,9.58000000000000007105427357601,200,0.100000000000000005551115123126,142,400,0,4,0.548793227430632080121597482503,leaky_relu,normal
390,,,,,,,,,,,390_0,RUNNING,BOTORCH_MODULAR,,75,0.000100000000000000004792173602,8,3112,0,1,0,leaky_relu,normal
391,,,,,,,,,,,216_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
392,,,,,,,,,,,392_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,1,leaky_relu,normal
393,,,,,,,,,,,393_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0,4,0.579354341737467137818384799175,leaky_relu,normal
394,,,,,,,,,,,394_0,RUNNING,BOTORCH_MODULAR,,76,0.000100000000000000004792173602,8,2881,0,2,0,leaky_relu,normal
395,,,,,,,,,,,395_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,3164,4799,0.5,1,0,leaky_relu,normal
396,,,,,,,,,,,396_0,RUNNING,BOTORCH_MODULAR,,184,0.000100000000000000004792173602,8,4155,0,1,0,leaky_relu,normal
397,,,,,,,,,,,397_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5263,0.5,4,0,leaky_relu,normal
398,,,,,,,,,,,398_0,RUNNING,BOTORCH_MODULAR,,93,0.000100000000000000004792173602,8,4884,0,1,0,leaky_relu,normal
399,1753326741,63,1753326804,1753330212,3408,python3 .tests/mnist/train --epochs 108 --learning_rate 0.0001 --batch_size 1921 --hidden_size 5359 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18600043,399_0,COMPLETED,BOTORCH_MODULAR,98.39000000000000056843418860808,108,0.000100000000000000004792173602,1921,5359,0.5,1,0,leaky_relu,normal
400,,,,,,,,,,,400_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,0.384150673543195375625458609647,leaky_relu,normal
401,,,,,,,,,,,401_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5908,0.5,1,0,leaky_relu,normal
402,,,,,,,,,,,402_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0.22975800978066512891828665488,leaky_relu,normal
403,,,,,,,,,,,403_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,2,0,leaky_relu,normal
404,,,,,,,,,,,404_0,RUNNING,BOTORCH_MODULAR,,168,0.000100000000000000004792173602,4096,4709,0.5,1,0,leaky_relu,normal
405,,,,,,,,,,,405_0,FAILED,BOTORCH_MODULAR,,43,0.000100000000000000004792173602,2991,7887,0,4,0.492991560110255411242263789973,leaky_relu,normal
406,1753327220,28,1753327248,1753327686,438,python3 .tests/mnist/train --epochs 10 --learning_rate 0.10000000000000000555 --batch_size 4096 --hidden_size 8192 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1720,18600069,406_0,COMPLETED,BOTORCH_MODULAR,10.27999999999999936051153781591,10,0.100000000000000005551115123126,4096,8192,0.5,1,0,leaky_relu,normal
407,,,,,,,,,,,407_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4017,0.5,1,0,leaky_relu,normal
408,,,,,,,,,,,408_0,FAILED,BOTORCH_MODULAR,,196,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
409,,,,,,,,,,,209_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
410,,,,,,,,,,,410_0,FAILED,BOTORCH_MODULAR,,144,0.000100000000000000004792173602,4096,5703,0.5,1,0,leaky_relu,normal
411,1753327222,26,1753327248,1753327545,297,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 8 --hidden_size 119 --dropout 0 --activation leaky_relu --num_dense_layers 4 --init normal --weight_decay 0,0,,n1720,18600071,411_0,COMPLETED,BOTORCH_MODULAR,95.769999999999996020960679743439,10,0.000100000000000000004792173602,8,119,0,4,0,leaky_relu,normal
412,,,,,,,,,,,412_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
413,,,,,,,,,,,413_0,FAILED,BOTORCH_MODULAR,,146,0.000100000000000000004792173602,8,5056,0.5,1,0,leaky_relu,normal
414,,,,,,,,,,,414_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1265,6747,0.5,1,0,leaky_relu,normal
415,,,,,,,,,,,415_0,FAILED,BOTORCH_MODULAR,,155,0.100000000000000005551115123126,3844,8047,0,4,0.187863562826283636564284051929,leaky_relu,normal
416,,,,,,,,,,,416_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1436,8192,0,4,0.520917073415104958655774680665,leaky_relu,normal
417,,,,,,,,,,,417_0,FAILED,BOTORCH_MODULAR,,97,0.000100000000000000004792173602,8,4087,0,1,0,leaky_relu,normal
418,,,,,,,,,,,418_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4926,0.5,1,0,leaky_relu,normal
419,,,,,,,,,,,419_0,FAILED,BOTORCH_MODULAR,,106,0.000100000000000000004792173602,8,3954,0.5,1,0,leaky_relu,normal
420,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
421,,,,,,,,,,,421_0,FAILED,BOTORCH_MODULAR,,188,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
422,,,,,,,,,,,422_0,FAILED,BOTORCH_MODULAR,,86,0.000100000000000000004792173602,8,5019,0,2,0,leaky_relu,normal
423,,,,,,,,,,,423_0,FAILED,BOTORCH_MODULAR,,140,0.000100000000000000004792173602,133,8192,0.43117310450071205663036266742,1,0,leaky_relu,normal
424,,,,,,,,,,,424_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4648,0.5,1,0,leaky_relu,normal
425,,,,,,,,,,,425_0,FAILED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0,4,1,leaky_relu,normal
426,,,,,,,,,,,426_0,FAILED,BOTORCH_MODULAR,,78,0.000100000000000000004792173602,8,2345,0.5,3,0,leaky_relu,normal
427,,,,,,,,,,,427_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4920,0,4,0,leaky_relu,normal
428,,,,,,,,,,,428_0,FAILED,BOTORCH_MODULAR,,74,0.000100000000000000004792173602,8,3021,0,2,0,leaky_relu,normal
429,,,,,,,,,,,429_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4529,0.5,4,0,leaky_relu,normal
430,,,,,,,,,,,430_0,FAILED,BOTORCH_MODULAR,,93,0.000100000000000000004792173602,8,3595,0,1,0,leaky_relu,normal
431,,,,,,,,,,,431_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4730,0.5,1,0,leaky_relu,normal
432,,,,,,,,,,,432_0,FAILED,BOTORCH_MODULAR,,118,0.000100000000000000004792173602,8,5195,0.5,4,0,leaky_relu,normal
433,,,,,,,,,,,372_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
434,,,,,,,,,,,434_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3885,0,3,0,leaky_relu,normal
435,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
436,,,,,,,,,,,436_0,FAILED,BOTORCH_MODULAR,,134,0.000100000000000000004792173602,8,2006,0,4,0,leaky_relu,normal
437,,,,,,,,,,,437_0,FAILED,BOTORCH_MODULAR,,145,0.000100000000000000004792173602,8,4563,0.5,1,0,leaky_relu,normal
438,,,,,,,,,,,438_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1303,7283,0.5,1,0,leaky_relu,normal
439,,,,,,,,,,,439_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3425,0,2,0,leaky_relu,normal
440,,,,,,,,,,,440_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4257,0.5,2,0,leaky_relu,normal
441,,,,,,,,,,,441_0,FAILED,BOTORCH_MODULAR,,76,0.000100000000000000004792173602,8,3146,0.5,2,0,leaky_relu,normal
442,,,,,,,,,,,442_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,3144,5190,0.5,1,0,leaky_relu,normal
443,,,,,,,,,,,443_0,FAILED,BOTORCH_MODULAR,,71,0.000100000000000000004792173602,8,4250,0,3,0,leaky_relu,normal
444,,,,,,,,,,,444_0,FAILED,BOTORCH_MODULAR,,188,0.000100000000000000004792173602,8,8,0.5,4,0,leaky_relu,normal
445,,,,,,,,,,,392_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,1,leaky_relu,normal
446,,,,,,,,,,,446_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3374,0,4,0,leaky_relu,normal
447,,,,,,,,,,,447_0,FAILED,BOTORCH_MODULAR,,102,0.000100000000000000004792173602,8,3749,0.5,4,0,leaky_relu,normal
448,,,,,,,,,,,448_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,7652,0,4,0,leaky_relu,normal
449,,,,,,,,,,,449_0,FAILED,BOTORCH_MODULAR,,65,0.000100000000000000004792173602,8,4257,0,4,0,leaky_relu,normal
450,,,,,,,,,,,450_0,FAILED,BOTORCH_MODULAR,,198,0.000100000000000000004792173602,951,4286,0.5,1,0,leaky_relu,normal
451,,,,,,,,,,,451_0,FAILED,BOTORCH_MODULAR,,167,0.000100000000000000004792173602,3224,4393,0.5,1,0,leaky_relu,normal
452,,,,,,,,,,,452_0,FAILED,BOTORCH_MODULAR,,104,0.000100000000000000004792173602,8,5982,0,1,0,leaky_relu,normal
453,,,,,,,,,,,453_0,RUNNING,BOTORCH_MODULAR,,178,0.000100000000000000004792173602,8,7455,0.5,1,0,leaky_relu,normal
454,,,,,,,,,,,454_0,FAILED,BOTORCH_MODULAR,,79,0.000100000000000000004792173602,8,2310,0,1,0,leaky_relu,normal
455,,,,,,,,,,,455_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4484,0.070813222078610904564044403742,4,0,leaky_relu,normal
456,,,,,,,,,,,191_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,1,leaky_relu,normal
457,,,,,,,,,,,457_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,4363,0.5,4,0,leaky_relu,normal
458,,,,,,,,,,,458_0,FAILED,BOTORCH_MODULAR,,92,0.000100000000000000004792173602,851,6751,0.5,1,0,leaky_relu,normal
459,,,,,,,,,,,206_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
460,,,,,,,,,,,460_0,RUNNING,BOTORCH_MODULAR,,62,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
461,,,,,,,,,,,216_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
462,,,,,,,,,,,462_0,FAILED,BOTORCH_MODULAR,,93,0.000100000000000000004792173602,8,3442,0.5,1,0,leaky_relu,normal
463,,,,,,,,,,,463_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1110,7724,0,1,0,leaky_relu,normal
464,,,,,,,,,,,464_0,RUNNING,BOTORCH_MODULAR,,115,0.000100000000000000004792173602,2974,4935,0.5,1,0,leaky_relu,normal
465,,,,,,,,,,,465_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4163,0.5,3,0,leaky_relu,normal
466,,,,,,,,,,,466_0,RUNNING,BOTORCH_MODULAR,,64,0.000100000000000000004792173602,8,7513,0.5,2,0,leaky_relu,normal
467,,,,,,,,,,,467_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2133,5122,0.5,1,0,leaky_relu,normal
468,,,,,,,,,,,468_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,2232,0.5,4,0,leaky_relu,normal
469,,,,,,,,,,,469_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,2791,8192,0,4,1,leaky_relu,normal
470,,,,,,,,,,,470_0,RUNNING,BOTORCH_MODULAR,,155,0.000100000000000000004792173602,4096,6185,0.5,1,0,leaky_relu,normal
471,,,,,,,,,,,471_0,RUNNING,BOTORCH_MODULAR,,119,0.000100000000000000004792173602,8,3318,0.5,2,0,leaky_relu,normal
472,,,,,,,,,,,412_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
473,,,,,,,,,,,473_0,RUNNING,BOTORCH_MODULAR,,10,0.100000000000000005551115123126,8,8192,0.5,4,1,leaky_relu,normal
474,,,,,,,,,,,474_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,215,7746,0.5,4,0.523900012977612017373019170918,leaky_relu,normal
475,,,,,,,,,,,153_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,8192,0.5,1,0,leaky_relu,normal
476,,,,,,,,,,,476_0,RUNNING,BOTORCH_MODULAR,,97,0.000100000000000000004792173602,8,573,0,3,0,leaky_relu,normal
477,,,,,,,,,,,477_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4718,0,4,0,leaky_relu,normal
478,,,,,,,,,,,478_0,RUNNING,BOTORCH_MODULAR,,74,0.000100000000000000004792173602,8,3266,0,2,0,leaky_relu,normal
479,,,,,,,,,,,479_0,RUNNING,BOTORCH_MODULAR,,95,0.000100000000000000004792173602,8,3551,0,1,0,leaky_relu,normal
480,,,,,,,,,,,480_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4227,0.5,3,0,leaky_relu,normal
481,,,,,,,,,,,481_0,RUNNING,BOTORCH_MODULAR,,81,0.000100000000000000004792173602,8,4516,0.5,4,0,leaky_relu,normal
482,,,,,,,,,,,482_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4664,0.5,1,0,leaky_relu,normal
483,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
484,,,,,,,,,,,484_0,RUNNING,BOTORCH_MODULAR,,106,0.000100000000000000004792173602,8,2564,0,3,0,leaky_relu,normal
485,,,,,,,,,,,485_0,FAILED,BOTORCH_MODULAR,,94,0.000100000000000000004792173602,8,1759,0.5,4,0,leaky_relu,normal
486,,,,,,,,,,,486_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4902,0,4,0,leaky_relu,normal
487,,,,,,,,,,,487_0,FAILED,BOTORCH_MODULAR,,70,0.000100000000000000004792173602,8,3339,0,2,0,leaky_relu,normal
488,,,,,,,,,,,488_0,FAILED,BOTORCH_MODULAR,,127,0.000100000000000000004792173602,8,4264,0.5,1,0,leaky_relu,normal
489,,,,,,,,,,,489_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4303,0.5,4,0,leaky_relu,normal
490,,,,,,,,,,,490_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,4854,0.5,1,0,leaky_relu,normal
491,,,,,,,,,,,218_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0.5,4,0,leaky_relu,normal
492,,,,,,,,,,,492_0,FAILED,BOTORCH_MODULAR,,87,0.000100000000000000004792173602,8,3614,0,1,0,leaky_relu,normal
493,,,,,,,,,,,493_0,FAILED,BOTORCH_MODULAR,,102,0.000100000000000000004792173602,8,3613,0,3,0,leaky_relu,normal
494,,,,,,,,,,,494_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3811,0.5,3,0,leaky_relu,normal
495,,,,,,,,,,,151_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,1,0,leaky_relu,normal
496,,,,,,,,,,,496_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3789,0,3,0,leaky_relu,normal
497,,,,,,,,,,,313_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
498,,,,,,,,,,,498_0,FAILED,BOTORCH_MODULAR,,176,0.000100000000000000004792173602,8,166,0.5,4,0,leaky_relu,normal
499,,,,,,,,,,,499_0,FAILED,BOTORCH_MODULAR,,99,0.000100000000000000004792173602,8,4461,0.5,4,0,leaky_relu,normal
500,,,,,,,,,,,372_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
501,,,,,,,,,,,501_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2548,5135,0.5,1,0,leaky_relu,normal
502,,,,,,,,,,,502_0,FAILED,BOTORCH_MODULAR,,96,0.000100000000000000004792173602,8,3345,0.5,2,0,leaky_relu,normal
503,,,,,,,,,,,503_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,2,0,leaky_relu,normal
504,,,,,,,,,,,504_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,534,5549,0.5,1,0,leaky_relu,normal
505,,,,,,,,,,,505_0,FAILED,BOTORCH_MODULAR,,103,0.000100000000000000004792173602,8,8143,0.5,3,0,leaky_relu,normal
506,,,,,,,,,,,506_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,5727,0,1,0,leaky_relu,normal
507,,,,,,,,,,,507_0,FAILED,BOTORCH_MODULAR,,81,0.000100000000000000004792173602,8,1718,0,3,0,leaky_relu,normal
508,1753346872,951,1753347823,1753350502,2679,python3 .tests/mnist/train --epochs 154 --learning_rate 0.0001 --batch_size 3497 --hidden_size 4858 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1524,18601057,508_0,COMPLETED,BOTORCH_MODULAR,98.46999999999999886313162278384,154,0.000100000000000000004792173602,3497,4858,0.5,1,0,leaky_relu,normal
509,1753346874,948,1753347822,1753350366,2544,python3 .tests/mnist/train --epochs 103 --learning_rate 0.0001 --batch_size 3193 --hidden_size 5281 --dropout 0.5 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1523,18601058,509_0,COMPLETED,BOTORCH_MODULAR,98.319999999999993178789736703038,103,0.000100000000000000004792173602,3193,5281,0.5,1,0,leaky_relu,normal
510,,,,,,,,,,,510_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,1104,8192,0.5,4,0,leaky_relu,normal
511,,,,,,,,,,,511_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,1850,0,4,0,leaky_relu,normal
512,,,,,,,,,,,512_0,FAILED,BOTORCH_MODULAR,,78,0.000100000000000000004792173602,8,3723,0.5,1,0,leaky_relu,normal
513,,,,,,,,,,,513_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3240,0,4,0,leaky_relu,normal
514,,,,,,,,,,,514_0,FAILED,BOTORCH_MODULAR,,64,0.000100000000000000004792173602,8,4627,0,3,0,leaky_relu,normal
515,,,,,,,,,,,515_0,FAILED,BOTORCH_MODULAR,,70,0.000100000000000000004792173602,8,1947,0,2,0,leaky_relu,normal
516,,,,,,,,,,,516_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3189,0,2,0,leaky_relu,normal
517,,,,,,,,,,,517_0,FAILED,BOTORCH_MODULAR,,67,0.000100000000000000004792173602,8,3522,0,4,0,leaky_relu,normal
518,,,,,,,,,,,518_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,6630,0.5,1,0,leaky_relu,normal
519,,,,,,,,,,,519_0,FAILED,BOTORCH_MODULAR,,175,0.000100000000000000004792173602,1706,4209,0.5,1,0,leaky_relu,normal
520,,,,,,,,,,,520_0,FAILED,BOTORCH_MODULAR,,111,0.000100000000000000004792173602,193,5471,0,1,0,leaky_relu,normal
521,,,,,,,,,,,521_0,FAILED,BOTORCH_MODULAR,,78,0.000100000000000000004792173602,8,6758,0,4,0,leaky_relu,normal
522,,,,,,,,,,,522_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,6128,0.5,4,0,leaky_relu,normal
523,,,,,,,,,,,523_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8171,0.5,3,0,leaky_relu,normal
524,,,,,,,,,,,524_0,FAILED,BOTORCH_MODULAR,,94,0.000100000000000000004792173602,8,2977,0,1,0,leaky_relu,normal
525,,,,,,,,,,,392_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,1,leaky_relu,normal
526,,,,,,,,,,,216_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,1,0,leaky_relu,normal
527,,,,,,,,,,,527_0,FAILED,BOTORCH_MODULAR,,132,0.000100000000000000004792173602,4096,6244,0.5,1,0,leaky_relu,normal
528,,,,,,,,,,,528_0,FAILED,BOTORCH_MODULAR,,76,0.000100000000000000004792173602,8,3064,0.5,3,0,leaky_relu,normal
529,,,,,,,,,,,529_0,FAILED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,3473,3988,0.5,1,0,leaky_relu,normal
530,,,,,,,,,,,530_0,FAILED,BOTORCH_MODULAR,,173,0.000100000000000000004792173602,8,4579,0.5,3,0,leaky_relu,normal
531,,,,,,,,,,,531_0,FAILED,BOTORCH_MODULAR,,62,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
532,,,,,,,,,,,532_0,FAILED,BOTORCH_MODULAR,,90,0.000100000000000000004792173602,8,1261,0.5,3,0,leaky_relu,normal
533,,,,,,,,,,,533_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0.286234240915621662715295769885,leaky_relu,normal
534,1753347617,203,1753347820,1753349337,1517,python3 .tests/mnist/train --epochs 56 --learning_rate 0.0001 --batch_size 827 --hidden_size 5340 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1255,18601101,534_0,COMPLETED,BOTORCH_MODULAR,98.299999999999997157829056959599,56,0.000100000000000000004792173602,827,5340,0,1,0,leaky_relu,normal
535,1753347627,659,1753348286,1753350394,2108,python3 .tests/mnist/train --epochs 118 --learning_rate 0.0001 --batch_size 1232 --hidden_size 5411 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1528,18601108,535_0,COMPLETED,BOTORCH_MODULAR,98.349999999999994315658113919199,118,0.000100000000000000004792173602,1232,5411,0,1,0,leaky_relu,normal
536,,,,,,,,,,,536_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4650,0,4,0,leaky_relu,normal
537,,,,,,,,,,,537_0,FAILED,BOTORCH_MODULAR,,70,0.000100000000000000004792173602,8,3312,0,2,0,leaky_relu,normal
538,,,,,,,,,,,538_0,FAILED,BOTORCH_MODULAR,,81,0.000100000000000000004792173602,8,3294,0.5,3,0,leaky_relu,normal
539,,,,,,,,,,,539_0,FAILED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5103,0.5,4,0,leaky_relu,normal
540,,,,,,,,,,,313_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
541,1753347932,354,1753348286,1753348580,294,python3 .tests/mnist/train --epochs 10 --learning_rate 0.0001 --batch_size 1176 --hidden_size 8192 --dropout 0 --activation leaky_relu --num_dense_layers 1 --init normal --weight_decay 0,0,,n1528,18601122,541_0,COMPLETED,BOTORCH_MODULAR,96.900000000000005684341886080801,10,0.000100000000000000004792173602,1176,8192,0,1,0,leaky_relu,normal
542,,,,,,,,,,,542_0,RUNNING,BOTORCH_MODULAR,,167,0.000100000000000000004792173602,1387,3234,0,1,0,leaky_relu,normal
543,,,,,,,,,,,543_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4558,0,4,0,leaky_relu,normal
544,,,,,,,,,,,544_0,RUNNING,BOTORCH_MODULAR,,76,0.000100000000000000004792173602,8,2683,0,3,0,leaky_relu,normal
545,,,,,,,,,,,545_0,RUNNING,BOTORCH_MODULAR,,97,0.000100000000000000004792173602,8,4198,0.5,4,0,leaky_relu,normal
546,,,,,,,,,,,546_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,7526,0.5,4,0,leaky_relu,normal
547,,,,,,,,,,,547_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3652,0,3,0,leaky_relu,normal
548,,,,,,,,,,,548_0,RUNNING,BOTORCH_MODULAR,,97,0.000100000000000000004792173602,8,3764,0.5,1,0,leaky_relu,normal
549,,,,,,,,,,,549_0,RUNNING,BOTORCH_MODULAR,,69,0.000100000000000000004792173602,8,2100,0,2,0,leaky_relu,normal
550,,,,,,,,,,,550_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,66,4680,0.5,1,0,leaky_relu,normal
551,,,,,,,,,,,372_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
552,,,,,,,,,,,552_0,RUNNING,BOTORCH_MODULAR,,89,0.000100000000000000004792173602,8,2669,0,1,0,leaky_relu,normal
553,,,,,,,,,,,553_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0,1,0.178658659120983970280960306809,leaky_relu,normal
554,,,,,,,,,,,554_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4296,0.5,4,0,leaky_relu,normal
555,,,,,,,,,,,555_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3157,0,2,0,leaky_relu,normal
556,,,,,,,,,,,556_0,RUNNING,BOTORCH_MODULAR,,83,0.000100000000000000004792173602,8,3414,0,2,0,leaky_relu,normal
557,,,,,,,,,,,557_0,RUNNING,BOTORCH_MODULAR,,172,0.000100000000000000004792173602,8,1340,0.5,4,0,leaky_relu,normal
558,,,,,,,,,,,558_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2470,4654,0.5,1,0,leaky_relu,normal
559,,,,,,,,,,,559_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,3637,0.5,4,0,leaky_relu,normal
560,,,,,,,,,,,313_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
561,,,,,,,,,,,206_0,ABANDONED,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,0,leaky_relu,normal
562,,,,,,,,,,,562_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,117,8192,0.480767104889141150270148727941,1,0,leaky_relu,normal
563,,,,,,,,,,,563_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,2591,8192,0.5,1,0,leaky_relu,normal
564,,,,,,,,,,,392_0,ABANDONED,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,8,8192,0.5,4,1,leaky_relu,normal
565,,,,,,,,,,,565_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,935,0,4,0,leaky_relu,normal
566,,,,,,,,,,,566_0,RUNNING,BOTORCH_MODULAR,,85,0.000100000000000000004792173602,8,1381,0.5,3,0,leaky_relu,normal
567,,,,,,,,,,,567_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,4096,5079,0.5,1,0,leaky_relu,normal
568,,,,,,,,,,,568_0,RUNNING,BOTORCH_MODULAR,,147,0.000100000000000000004792173602,8,8,0.5,4,0,leaky_relu,normal
569,,,,,,,,,,,569_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5508,0,3,0,leaky_relu,normal
570,,,,,,,,,,,570_0,RUNNING,BOTORCH_MODULAR,,58,0.000100000000000000004792173602,8,4392,0.5,3,0,leaky_relu,normal
571,,,,,,,,,,,571_0,RUNNING,BOTORCH_MODULAR,,63,0.000100000000000000004792173602,8,4684,0,4,0,leaky_relu,normal
572,,,,,,,,,,,572_0,RUNNING,BOTORCH_MODULAR,,99,0.000100000000000000004792173602,8,3331,0,1,0,leaky_relu,normal
573,,,,,,,,,,,573_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0.5,4,1,leaky_relu,normal
574,,,,,,,,,,,574_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1399,4317,0.5,1,0,leaky_relu,normal
575,,,,,,,,,,,575_0,RUNNING,BOTORCH_MODULAR,,102,0.000100000000000000004792173602,8,195,0,3,0,leaky_relu,normal
576,,,,,,,,,,,576_0,RUNNING,BOTORCH_MODULAR,,110,0.000100000000000000004792173602,8,1417,0,4,0,leaky_relu,normal
577,,,,,,,,,,,342_0,ABANDONED,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,8,8192,0.5,4,1,leaky_relu,normal
578,,,,,,,,,,,578_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,7641,0.5,4,0.329700399357083340223795175916,leaky_relu,normal
579,,,,,,,,,,,579_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,5686,0,4,0,leaky_relu,normal
580,,,,,,,,,,,580_0,RUNNING,BOTORCH_MODULAR,,200,0.000100000000000000004792173602,1835,8192,0.5,4,1,leaky_relu,normal
581,,,,,,,,,,,581_0,RUNNING,BOTORCH_MODULAR,,200,0.100000000000000005551115123126,4096,8192,0.5,4,0.675484005164332490522838270408,leaky_relu,normal
582,,,,,,,,,,,582_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3818,0.5,3,0,leaky_relu,normal
583,,,,,,,,,,,583_0,RUNNING,BOTORCH_MODULAR,,77,0.000100000000000000004792173602,8,1946,0.5,4,0,leaky_relu,normal
584,,,,,,,,,,,584_0,RUNNING,BOTORCH_MODULAR,,136,0.000100000000000000004792173602,8,4900,0,4,0,leaky_relu,normal
585,,,,,,,,,,,585_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,4096,8192,0,1,1,leaky_relu,normal
586,,,,,,,,,,,586_0,RUNNING,BOTORCH_MODULAR,,136,0.000100000000000000004792173602,8,3729,0.5,3,0,leaky_relu,normal
587,,,,,,,,,,,587_0,RUNNING,BOTORCH_MODULAR,,107,0.000100000000000000004792173602,2840,8192,0.5,1,0,leaky_relu,normal
588,,,,,,,,,,,588_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,2513,0.5,4,0,leaky_relu,normal
589,,,,,,,,,,,589_0,RUNNING,BOTORCH_MODULAR,,69,0.000100000000000000004792173602,8,8192,0,4,0,leaky_relu,normal
590,,,,,,,,,,,590_0,RUNNING,BOTORCH_MODULAR,,133,0.000100000000000000004792173602,958,3832,0.5,1,0,leaky_relu,normal
591,,,,,,,,,,,591_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,2000,8192,0.5,4,0.350028498075888316876103090181,leaky_relu,normal
592,,,,,,,,,,,592_0,RUNNING,BOTORCH_MODULAR,,115,0.000100000000000000004792173602,4096,4772,0.5,1,0,leaky_relu,normal
593,,,,,,,,,,,593_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,4469,0,4,0,leaky_relu,normal
594,,,,,,,,,,,594_0,RUNNING,BOTORCH_MODULAR,,68,0.000100000000000000004792173602,8,2990,0,2,0,leaky_relu,normal
595,,,,,,,,,,,595_0,RUNNING,BOTORCH_MODULAR,,78,0.000100000000000000004792173602,8,3941,0.5,4,0,leaky_relu,normal
596,,,,,,,,,,,596_0,RUNNING,BOTORCH_MODULAR,,10,0.000100000000000000004792173602,8,3838,0.5,3,0,leaky_relu,normal

Copy raw data to clipboard

Download »results.csv« as file

Copy raw data to clipboard

Download »oo_errors.txt« as file

⚠ Job 18592320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,243) - Starting with JobEnvironment(job_id=18592320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_submitted.pkl
slurmstepd: error: *** JOB 18592320 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***



⚠ Job 18592312 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:04,206) - Starting with JobEnvironment(job_id=18592312, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:04,207) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_submitted.pkl
slurmstepd: error: *** JOB 18592312 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***



⚠ Job 18592310 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,847) - Starting with JobEnvironment(job_id=18592310, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,849) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_submitted.pkl
slurmstepd: error: *** JOB 18592310 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***



⚠ Job 18592327 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,384) - Starting with JobEnvironment(job_id=18592327, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,387) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_submitted.pkl
slurmstepd: error: *** JOB 18592327 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592366 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592366, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_submitted.pkl
slurmstepd: error: *** JOB 18592366 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592331 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,112) - Starting with JobEnvironment(job_id=18592331, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,114) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_submitted.pkl
slurmstepd: error: *** JOB 18592331 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592354 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,224) - Starting with JobEnvironment(job_id=18592354, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,226) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_submitted.pkl
slurmstepd: error: *** JOB 18592354 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***



⚠ Job 18592316 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,823) - Starting with JobEnvironment(job_id=18592316, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,824) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_submitted.pkl
slurmstepd: error: *** JOB 18592316 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***



⚠ Job 18592351 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,224) - Starting with JobEnvironment(job_id=18592351, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,226) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_submitted.pkl
slurmstepd: error: *** JOB 18592351 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***



⚠ Job 18592336 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,153) - Starting with JobEnvironment(job_id=18592336, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,155) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_submitted.pkl
slurmstepd: error: *** JOB 18592336 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592353 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,226) - Starting with JobEnvironment(job_id=18592353, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,227) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_submitted.pkl
slurmstepd: error: *** JOB 18592353 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***



⚠ Job 18592314 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,823) - Starting with JobEnvironment(job_id=18592314, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,825) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_submitted.pkl
slurmstepd: error: *** JOB 18592314 ON n1720 CANCELLED AT 2025-07-23T14:45:12 DUE TO TIME LIMIT ***



⚠ Job 18592367 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592367, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_submitted.pkl
slurmstepd: error: *** JOB 18592367 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592308 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,907) - Starting with JobEnvironment(job_id=18592308, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,909) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_submitted.pkl
slurmstepd: error: *** JOB 18592308 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***



⚠ Job 18592332 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,153) - Starting with JobEnvironment(job_id=18592332, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,156) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_submitted.pkl
slurmstepd: error: *** JOB 18592332 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592361 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592361, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_submitted.pkl
slurmstepd: error: *** JOB 18592361 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592317 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,846) - Starting with JobEnvironment(job_id=18592317, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,848) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_submitted.pkl
slurmstepd: error: *** JOB 18592317 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***



⚠ Job 18592341 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,083) - Starting with JobEnvironment(job_id=18592341, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_submitted.pkl
slurmstepd: error: *** JOB 18592341 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***



⚠ Job 18592339 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,085) - Starting with JobEnvironment(job_id=18592339, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_submitted.pkl
slurmstepd: error: *** JOB 18592339 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***



⚠ Job 18592330 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,383) - Starting with JobEnvironment(job_id=18592330, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,385) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_submitted.pkl
slurmstepd: error: *** JOB 18592330 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_submitted.pkl
slurmstepd: error: *** JOB 18592319 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***



⚠ Job 18592363 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592363, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,273) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_submitted.pkl
slurmstepd: error: *** JOB 18592363 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592355 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,206) - Starting with JobEnvironment(job_id=18592355, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_submitted.pkl
slurmstepd: error: *** JOB 18592355 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***



⚠ Job 18592323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,243) - Starting with JobEnvironment(job_id=18592323, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_submitted.pkl
slurmstepd: error: *** JOB 18592323 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***



⚠ Job 18592344 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,084) - Starting with JobEnvironment(job_id=18592344, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_submitted.pkl
slurmstepd: error: *** JOB 18592344 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***



⚠ Job 18592322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592322, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,246) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_submitted.pkl
slurmstepd: error: *** JOB 18592322 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***



⚠ Job 18592338 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,085) - Starting with JobEnvironment(job_id=18592338, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_submitted.pkl
slurmstepd: error: *** JOB 18592338 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***



⚠ Job 18592329 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,384) - Starting with JobEnvironment(job_id=18592329, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,387) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_submitted.pkl
slurmstepd: error: *** JOB 18592329 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592318 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592318, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,246) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_submitted.pkl
slurmstepd: error: *** JOB 18592318 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***



⚠ Job 18592333 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,383) - Starting with JobEnvironment(job_id=18592333, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,385) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_submitted.pkl
slurmstepd: error: *** JOB 18592333 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592340 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,084) - Starting with JobEnvironment(job_id=18592340, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,088) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_submitted.pkl
slurmstepd: error: *** JOB 18592340 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***



⚠ Job 18592334 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,381) - Starting with JobEnvironment(job_id=18592334, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,386) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_submitted.pkl
slurmstepd: error: *** JOB 18592334 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***



⚠ Job 18592364 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592364, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,273) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_submitted.pkl
slurmstepd: error: *** JOB 18592364 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592370 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:49:07,852) - Starting with JobEnvironment(job_id=18592370, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:49:07,855) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_submitted.pkl
slurmstepd: error: *** JOB 18592370 ON n1720 CANCELLED AT 2025-07-23T14:49:11 DUE TO TIME LIMIT ***



⚠ Job 18592368 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592368, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_submitted.pkl
slurmstepd: error: *** JOB 18592368 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592369 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592369, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_submitted.pkl
slurmstepd: error: *** JOB 18592369 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592375 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:49:38,508) - Starting with JobEnvironment(job_id=18592375, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:49:38,510) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_submitted.pkl
slurmstepd: error: *** JOB 18592375 ON n1720 CANCELLED AT 2025-07-23T14:49:41 DUE TO TIME LIMIT ***



⚠ Job 18594392 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594392, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_submitted.pkl
slurmstepd: error: *** JOB 18594392 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***



⚠ Job 18594393 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594393, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,991) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_submitted.pkl
slurmstepd: error: *** JOB 18594393 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***



⚠ Job 18594403 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,789) - Starting with JobEnvironment(job_id=18594403, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_submitted.pkl
slurmstepd: error: *** JOB 18594403 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594400 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594400, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_submitted.pkl
slurmstepd: error: *** JOB 18594400 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***



⚠ Job 18594395 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594395, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_submitted.pkl
slurmstepd: error: *** JOB 18594395 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***



⚠ Job 18594401 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594401, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_submitted.pkl
slurmstepd: error: *** JOB 18594401 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594404 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594404, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,789) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_submitted.pkl
slurmstepd: error: *** JOB 18594404 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594402 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594402, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_submitted.pkl
slurmstepd: error: *** JOB 18594402 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594405 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594405, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,789) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_submitted.pkl
slurmstepd: error: *** JOB 18594405 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594406 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,789) - Starting with JobEnvironment(job_id=18594406, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_submitted.pkl
slurmstepd: error: *** JOB 18594406 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***



⚠ Job 18594485 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,258) - Starting with JobEnvironment(job_id=18594485, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,261) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_submitted.pkl
slurmstepd: error: *** JOB 18594485 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594502 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,633) - Starting with JobEnvironment(job_id=18594502, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,635) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_submitted.pkl
slurmstepd: error: *** JOB 18594502 ON n1719 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***



⚠ Job 18594603 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,710) - Starting with JobEnvironment(job_id=18594603, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,712) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_submitted.pkl
slurmstepd: error: *** JOB 18594603 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594491 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594491, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_submitted.pkl
slurmstepd: error: *** JOB 18594491 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594483 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:46,624) - Starting with JobEnvironment(job_id=18594483, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:46,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_submitted.pkl
slurmstepd: error: *** JOB 18594483 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594487 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594487, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_submitted.pkl
slurmstepd: error: *** JOB 18594487 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594588 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594588, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_submitted.pkl
slurmstepd: error: *** JOB 18594588 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***



⚠ Job 18594602 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594602, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,724) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_submitted.pkl
slurmstepd: error: *** JOB 18594602 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594585 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594585, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_submitted.pkl
slurmstepd: error: *** JOB 18594585 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***



⚠ Job 18594608 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,720) - Starting with JobEnvironment(job_id=18594608, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_submitted.pkl
slurmstepd: error: *** JOB 18594608 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594589 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594589, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_submitted.pkl
slurmstepd: error: *** JOB 18594589 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***



⚠ Job 18594552 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:23:18,421) - Starting with JobEnvironment(job_id=18594552, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:23:18,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_submitted.pkl
slurmstepd: error: *** JOB 18594552 ON n1719 CANCELLED AT 2025-07-23T16:23:15 DUE TO TIME LIMIT ***



⚠ Job 18594490 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594490, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_submitted.pkl
slurmstepd: error: *** JOB 18594490 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594574 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:14,428) - Starting with JobEnvironment(job_id=18594574, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:14,430) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_submitted.pkl
slurmstepd: error: *** JOB 18594574 ON n1719 CANCELLED AT 2025-07-23T16:24:16 DUE TO TIME LIMIT ***



⚠ Job 18594605 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594605, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_submitted.pkl
slurmstepd: error: *** JOB 18594605 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594600 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,721) - Starting with JobEnvironment(job_id=18594600, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_submitted.pkl
slurmstepd: error: *** JOB 18594600 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594506 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,757) - Starting with JobEnvironment(job_id=18594506, hostname=n1714, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_submitted.pkl
slurmstepd: error: *** JOB 18594506 ON n1714 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***



⚠ Job 18594488 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594488, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_submitted.pkl
slurmstepd: error: *** JOB 18594488 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594604 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594604, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_submitted.pkl
slurmstepd: error: *** JOB 18594604 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594486 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,258) - Starting with JobEnvironment(job_id=18594486, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_submitted.pkl
slurmstepd: error: *** JOB 18594486 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***



⚠ Job 18594607 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,721) - Starting with JobEnvironment(job_id=18594607, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_submitted.pkl
slurmstepd: error: *** JOB 18594607 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594551 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:23:13,431) - Starting with JobEnvironment(job_id=18594551, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:23:13,433) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_submitted.pkl
slurmstepd: error: *** JOB 18594551 ON n1719 CANCELLED AT 2025-07-23T16:23:15 DUE TO TIME LIMIT ***



⚠ Job 18594586 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594586, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,271) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_submitted.pkl
slurmstepd: error: *** JOB 18594586 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***



⚠ Job 18594665 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:26:47,443) - Starting with JobEnvironment(job_id=18594665, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:26:47,450) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_submitted.pkl
slurmstepd: error: *** JOB 18594665 ON n1719 CANCELLED AT 2025-07-23T16:27:15 DUE TO TIME LIMIT ***



⚠ Job 18594503 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,550) - Starting with JobEnvironment(job_id=18594503, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,552) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_submitted.pkl
slurmstepd: error: *** JOB 18594503 ON n1719 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***



⚠ Job 18597170 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:38,815) - Starting with JobEnvironment(job_id=18597170, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:38,817) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_submitted.pkl
slurmstepd: error: *** JOB 18597170 ON n1720 CANCELLED AT 2025-07-23T18:06:03 DUE TO TIME LIMIT ***



⚠ Job 18597159 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:06,971) - Starting with JobEnvironment(job_id=18597159, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:06,973) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_submitted.pkl
slurmstepd: error: *** JOB 18597159 ON n1720 CANCELLED AT 2025-07-23T18:05:32 DUE TO TIME LIMIT ***



⚠ Job 18597175 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,389) - Starting with JobEnvironment(job_id=18597175, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,391) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_submitted.pkl
slurmstepd: error: *** JOB 18597175 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18597129 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:28,896) - Starting with JobEnvironment(job_id=18597129, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:28,897) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_submitted.pkl
slurmstepd: error: *** JOB 18597129 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***



⚠ Job 18597116 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,590) - Starting with JobEnvironment(job_id=18597116, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_submitted.pkl
slurmstepd: error: *** JOB 18597116 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***



⚠ Job 18597128 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:29,698) - Starting with JobEnvironment(job_id=18597128, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:29,699) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_submitted.pkl
slurmstepd: error: *** JOB 18597128 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***



⚠ Job 18597122 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,589) - Starting with JobEnvironment(job_id=18597122, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_submitted.pkl
slurmstepd: error: *** JOB 18597122 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***



⚠ Job 18597224 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:22,909) - Starting with JobEnvironment(job_id=18597224, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:22,912) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_submitted.pkl
slurmstepd: error: *** JOB 18597224 ON n1720 CANCELLED AT 2025-07-23T18:08:33 DUE TO TIME LIMIT ***



⚠ Job 18597174 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,388) - Starting with JobEnvironment(job_id=18597174, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,389) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_submitted.pkl
slurmstepd: error: *** JOB 18597174 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18597126 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:28,924) - Starting with JobEnvironment(job_id=18597126, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:28,925) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_submitted.pkl
slurmstepd: error: *** JOB 18597126 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***



⚠ Job 18597232 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,605) - Starting with JobEnvironment(job_id=18597232, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_submitted.pkl
slurmstepd: error: *** JOB 18597232 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597176 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,389) - Starting with JobEnvironment(job_id=18597176, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,391) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_submitted.pkl
slurmstepd: error: *** JOB 18597176 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18597120 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,589) - Starting with JobEnvironment(job_id=18597120, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_submitted.pkl
slurmstepd: error: *** JOB 18597120 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***



⚠ Job 18597229 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,611) - Starting with JobEnvironment(job_id=18597229, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,612) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_submitted.pkl
slurmstepd: error: *** JOB 18597229 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597167 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:38,815) - Starting with JobEnvironment(job_id=18597167, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:38,817) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_submitted.pkl
slurmstepd: error: *** JOB 18597167 ON n1720 CANCELLED AT 2025-07-23T18:06:03 DUE TO TIME LIMIT ***



⚠ Job 18597157 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:04:33,940) - Starting with JobEnvironment(job_id=18597157, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:04:33,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_submitted.pkl
slurmstepd: error: *** JOB 18597157 ON n1720 CANCELLED AT 2025-07-23T18:04:32 DUE TO TIME LIMIT ***



⚠ Job 18597235 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:09:27,570) - Starting with JobEnvironment(job_id=18597235, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:09:27,571) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_submitted.pkl
slurmstepd: error: *** JOB 18597235 ON n1720 CANCELLED AT 2025-07-23T18:09:32 DUE TO TIME LIMIT ***



⚠ Job 18597228 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,615) - Starting with JobEnvironment(job_id=18597228, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_submitted.pkl
slurmstepd: error: *** JOB 18597228 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597222 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:49,817) - Starting with JobEnvironment(job_id=18597222, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:49,819) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_submitted.pkl
slurmstepd: error: *** JOB 18597222 ON n1720 CANCELLED AT 2025-07-23T18:08:02 DUE TO TIME LIMIT ***



⚠ Job 18597216 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:17,206) - Starting with JobEnvironment(job_id=18597216, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:17,209) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_submitted.pkl
slurmstepd: error: *** JOB 18597216 ON n1720 CANCELLED AT 2025-07-23T18:07:32 DUE TO TIME LIMIT ***



⚠ Job 18597227 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,609) - Starting with JobEnvironment(job_id=18597227, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_submitted.pkl
slurmstepd: error: *** JOB 18597227 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597230 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,601) - Starting with JobEnvironment(job_id=18597230, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,604) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_submitted.pkl
slurmstepd: error: *** JOB 18597230 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597213 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:44,831) - Starting with JobEnvironment(job_id=18597213, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:44,833) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_submitted.pkl
slurmstepd: error: *** JOB 18597213 ON n1720 CANCELLED AT 2025-07-23T18:07:02 DUE TO TIME LIMIT ***



⚠ Job 18597177 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,396) - Starting with JobEnvironment(job_id=18597177, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,398) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_submitted.pkl
slurmstepd: error: *** JOB 18597177 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18597172 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,388) - Starting with JobEnvironment(job_id=18597172, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,390) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_submitted.pkl
slurmstepd: error: *** JOB 18597172 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18597217 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:18,103) - Starting with JobEnvironment(job_id=18597217, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:18,105) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_submitted.pkl
slurmstepd: error: *** JOB 18597217 ON n1720 CANCELLED AT 2025-07-23T18:07:32 DUE TO TIME LIMIT ***



⚠ Job 18597231 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,610) - Starting with JobEnvironment(job_id=18597231, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_submitted.pkl
slurmstepd: error: *** JOB 18597231 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597225 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:22,909) - Starting with JobEnvironment(job_id=18597225, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:22,912) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_submitted.pkl
slurmstepd: error: *** JOB 18597225 ON n1720 CANCELLED AT 2025-07-23T18:08:33 DUE TO TIME LIMIT ***



⚠ Job 18597212 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:44,832) - Starting with JobEnvironment(job_id=18597212, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:44,836) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_submitted.pkl
slurmstepd: error: *** JOB 18597212 ON n1720 CANCELLED AT 2025-07-23T18:07:02 DUE TO TIME LIMIT ***



⚠ Job 18597171 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,421) - Starting with JobEnvironment(job_id=18597171, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_submitted.pkl
slurmstepd: error: *** JOB 18597171 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***



⚠ Job 18598616 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,944) - Starting with JobEnvironment(job_id=18598616, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,945) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_submitted.pkl
slurmstepd: error: *** JOB 18598616 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***



⚠ Job 18598619 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,946) - Starting with JobEnvironment(job_id=18598619, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_submitted.pkl
slurmstepd: error: *** JOB 18598619 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***



⚠ Job 18598613 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,442) - Starting with JobEnvironment(job_id=18598613, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_submitted.pkl
slurmstepd: error: *** JOB 18598613 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***



⚠ Job 18598611 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,442) - Starting with JobEnvironment(job_id=18598611, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,444) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_submitted.pkl
slurmstepd: error: *** JOB 18598611 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***



⚠ Job 18598605 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,443) - Starting with JobEnvironment(job_id=18598605, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_submitted.pkl
slurmstepd: error: *** JOB 18598605 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***



⚠ Job 18598612 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,443) - Starting with JobEnvironment(job_id=18598612, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_submitted.pkl
slurmstepd: error: *** JOB 18598612 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***



⚠ Job 18598617 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,946) - Starting with JobEnvironment(job_id=18598617, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_submitted.pkl
slurmstepd: error: *** JOB 18598617 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***



⚠ Job 18598746 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,436) - Starting with JobEnvironment(job_id=18598746, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_submitted.pkl
slurmstepd: error: *** JOB 18598746 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***



⚠ Job 18598748 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,435) - Starting with JobEnvironment(job_id=18598748, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_submitted.pkl
slurmstepd: error: *** JOB 18598748 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***



⚠ Job 18598811 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,402) - Starting with JobEnvironment(job_id=18598811, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,404) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_submitted.pkl
slurmstepd: error: *** JOB 18598811 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***



⚠ Job 18598848 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:58,660) - Starting with JobEnvironment(job_id=18598848, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:58,661) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_submitted.pkl
slurmstepd: error: *** JOB 18598848 ON n1720 CANCELLED AT 2025-07-23T20:14:21 DUE TO TIME LIMIT ***



⚠ Job 18598745 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,436) - Starting with JobEnvironment(job_id=18598745, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_submitted.pkl
slurmstepd: error: *** JOB 18598745 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***



⚠ Job 18598739 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:07:40,642) - Starting with JobEnvironment(job_id=18598739, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:07:40,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_submitted.pkl
slurmstepd: error: *** JOB 18598739 ON n1720 CANCELLED AT 2025-07-23T20:07:50 DUE TO TIME LIMIT ***



⚠ Job 18598819 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,336) - Starting with JobEnvironment(job_id=18598819, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,338) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_submitted.pkl
slurmstepd: error: *** JOB 18598819 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***



⚠ Job 18598747 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,421) - Starting with JobEnvironment(job_id=18598747, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_submitted.pkl
slurmstepd: error: *** JOB 18598747 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***



⚠ Job 18598820 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:26,841) - Starting with JobEnvironment(job_id=18598820, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:26,843) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_submitted.pkl
slurmstepd: error: *** JOB 18598820 ON n1720 CANCELLED AT 2025-07-23T20:13:51 DUE TO TIME LIMIT ***



⚠ Job 18598821 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:26,742) - Starting with JobEnvironment(job_id=18598821, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:26,743) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_submitted.pkl
slurmstepd: error: *** JOB 18598821 ON n1720 CANCELLED AT 2025-07-23T20:13:51 DUE TO TIME LIMIT ***



⚠ Job 18598813 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,293) - Starting with JobEnvironment(job_id=18598813, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,295) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_submitted.pkl
slurmstepd: error: *** JOB 18598813 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***



⚠ Job 18598754 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:09:15,197) - Starting with JobEnvironment(job_id=18598754, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:09:15,200) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_submitted.pkl
slurmstepd: error: *** JOB 18598754 ON n1720 CANCELLED AT 2025-07-23T20:09:21 DUE TO TIME LIMIT ***



⚠ Job 18598851 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:17:06,853) - Starting with JobEnvironment(job_id=18598851, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:17:06,855) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_submitted.pkl
slurmstepd: error: *** JOB 18598851 ON n1720 CANCELLED AT 2025-07-23T20:17:22 DUE TO TIME LIMIT ***



⚠ Job 18599283 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,729) - Starting with JobEnvironment(job_id=18599283, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,732) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_submitted.pkl
slurmstepd: error: *** JOB 18599283 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599289 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,073) - Starting with JobEnvironment(job_id=18599289, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,075) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_submitted.pkl
slurmstepd: error: *** JOB 18599289 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599287 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,073) - Starting with JobEnvironment(job_id=18599287, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,076) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_submitted.pkl
slurmstepd: error: *** JOB 18599287 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599285 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,076) - Starting with JobEnvironment(job_id=18599285, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_submitted.pkl
slurmstepd: error: *** JOB 18599285 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599310 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599310, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_submitted.pkl
slurmstepd: error: *** JOB 18599310 ON n1720 CANCELLED AT 2025-07-23T22:31:37 DUE TO TIME LIMIT ***



⚠ Job 18599296 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599296, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_submitted.pkl
slurmstepd: error: *** JOB 18599296 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599293 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:27:20,757) - Starting with JobEnvironment(job_id=18599293, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:27:20,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_submitted.pkl
slurmstepd: error: *** JOB 18599293 ON n1720 CANCELLED AT 2025-07-23T22:27:35 DUE TO TIME LIMIT ***



⚠ Job 18599304 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599304, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_submitted.pkl
slurmstepd: error: *** JOB 18599304 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599308 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599308, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_submitted.pkl
slurmstepd: error: *** JOB 18599308 ON n1720 CANCELLED AT 2025-07-23T22:31:36 DUE TO TIME LIMIT ***



⚠ Job 18599279 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599279, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,522) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_submitted.pkl
slurmstepd: error: *** JOB 18599279 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599280 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599280, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_submitted.pkl
slurmstepd: error: *** JOB 18599280 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599295 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599295, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_submitted.pkl
slurmstepd: error: *** JOB 18599295 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599298 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599298, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_submitted.pkl
slurmstepd: error: *** JOB 18599298 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599309 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,802) - Starting with JobEnvironment(job_id=18599309, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_submitted.pkl
slurmstepd: error: *** JOB 18599309 ON n1720 CANCELLED AT 2025-07-23T22:31:37 DUE TO TIME LIMIT ***



⚠ Job 18599302 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,967) - Starting with JobEnvironment(job_id=18599302, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,970) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_submitted.pkl
slurmstepd: error: *** JOB 18599302 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599282 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599282, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_submitted.pkl
slurmstepd: error: *** JOB 18599282 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599277 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599277, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_submitted.pkl
slurmstepd: error: *** JOB 18599277 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599300 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599300, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_submitted.pkl
slurmstepd: error: *** JOB 18599300 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599307 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599307, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_submitted.pkl
slurmstepd: error: *** JOB 18599307 ON n1720 CANCELLED AT 2025-07-23T22:31:36 DUE TO TIME LIMIT ***



⚠ Job 18599305 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:30:40,574) - Starting with JobEnvironment(job_id=18599305, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:30:40,577) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_submitted.pkl
slurmstepd: error: *** JOB 18599305 ON n1720 CANCELLED AT 2025-07-23T22:30:35 DUE TO TIME LIMIT ***



⚠ Job 18599301 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599301, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_submitted.pkl
slurmstepd: error: *** JOB 18599301 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599306 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:30:40,574) - Starting with JobEnvironment(job_id=18599306, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:30:40,577) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_submitted.pkl
slurmstepd: error: *** JOB 18599306 ON n1720 CANCELLED AT 2025-07-23T22:30:35 DUE TO TIME LIMIT ***



⚠ Job 18599286 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,075) - Starting with JobEnvironment(job_id=18599286, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_submitted.pkl
slurmstepd: error: *** JOB 18599286 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599294 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:27:20,757) - Starting with JobEnvironment(job_id=18599294, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:27:20,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_submitted.pkl
slurmstepd: error: *** JOB 18599294 ON n1720 CANCELLED AT 2025-07-23T22:27:35 DUE TO TIME LIMIT ***



⚠ Job 18599290 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,075) - Starting with JobEnvironment(job_id=18599290, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,076) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_submitted.pkl
slurmstepd: error: *** JOB 18599290 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599292 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:26:49,167) - Starting with JobEnvironment(job_id=18599292, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:26:49,170) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_submitted.pkl
slurmstepd: error: *** JOB 18599292 ON n1720 CANCELLED AT 2025-07-23T22:27:05 DUE TO TIME LIMIT ***



⚠ Job 18599278 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599278, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,522) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_submitted.pkl
slurmstepd: error: *** JOB 18599278 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599284 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599284, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_submitted.pkl
slurmstepd: error: *** JOB 18599284 ON n1720 CANCELLED AT 2025-07-23T22:25:06 DUE TO TIME LIMIT ***



⚠ Job 18599288 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,072) - Starting with JobEnvironment(job_id=18599288, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,075) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_submitted.pkl
slurmstepd: error: *** JOB 18599288 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599281 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599281, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_submitted.pkl
slurmstepd: error: *** JOB 18599281 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599275 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599275, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_submitted.pkl
slurmstepd: error: *** JOB 18599275 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***



⚠ Job 18599297 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599297, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_submitted.pkl
slurmstepd: error: *** JOB 18599297 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***



⚠ Job 18599314 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,752) - Starting with JobEnvironment(job_id=18599314, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,754) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_submitted.pkl
slurmstepd: error: *** JOB 18599314 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599315 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599315, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_submitted.pkl
slurmstepd: error: *** JOB 18599315 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599316 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599316, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_submitted.pkl
slurmstepd: error: *** JOB 18599316 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599317 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599317, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_submitted.pkl
slurmstepd: error: *** JOB 18599317 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_submitted.pkl
slurmstepd: error: *** JOB 18599319 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_submitted.pkl
slurmstepd: error: *** JOB 18599320 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599321 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599321, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_submitted.pkl
slurmstepd: error: *** JOB 18599321 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***



⚠ Job 18599322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:26,435) - Starting with JobEnvironment(job_id=18599322, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:26,438) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_submitted.pkl
slurmstepd: error: *** JOB 18599322 ON n1720 CANCELLED AT 2025-07-23T22:34:36 DUE TO TIME LIMIT ***



⚠ Job 18599323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:31,599) - Starting with JobEnvironment(job_id=18599323, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:31,603) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_submitted.pkl
slurmstepd: error: *** JOB 18599323 ON n1720 CANCELLED AT 2025-07-23T22:34:36 DUE TO TIME LIMIT ***



⚠ Job 18599328 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,610) - Starting with JobEnvironment(job_id=18599328, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,613) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_submitted.pkl
slurmstepd: error: *** JOB 18599328 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***



⚠ Job 18599326 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,614) - Starting with JobEnvironment(job_id=18599326, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_submitted.pkl
slurmstepd: error: *** JOB 18599326 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***



⚠ Job 18599327 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,614) - Starting with JobEnvironment(job_id=18599327, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_submitted.pkl
slurmstepd: error: *** JOB 18599327 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***



⚠ Job 18599325 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:58,238) - Starting with JobEnvironment(job_id=18599325, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:58,241) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_submitted.pkl
slurmstepd: error: *** JOB 18599325 ON n1720 CANCELLED AT 2025-07-23T22:35:05 DUE TO TIME LIMIT ***



⚠ Job 18599324 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:59,285) - Starting with JobEnvironment(job_id=18599324, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:59,288) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_submitted.pkl
slurmstepd: error: *** JOB 18599324 ON n1720 CANCELLED AT 2025-07-23T22:35:05 DUE TO TIME LIMIT ***



⚠ Job 18599511 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599511, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_submitted.pkl
slurmstepd: error: *** JOB 18599511 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599510 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599510, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_submitted.pkl
slurmstepd: error: *** JOB 18599510 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599518 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599518, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_submitted.pkl
slurmstepd: error: *** JOB 18599518 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599513 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599513, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_submitted.pkl
slurmstepd: error: *** JOB 18599513 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599515 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599515, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_submitted.pkl
slurmstepd: error: *** JOB 18599515 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599524 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599524, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_submitted.pkl
slurmstepd: error: *** JOB 18599524 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***



⚠ Job 18599514 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599514, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_submitted.pkl
slurmstepd: error: *** JOB 18599514 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599516 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599516, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_submitted.pkl
slurmstepd: error: *** JOB 18599516 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599512 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599512, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_submitted.pkl
slurmstepd: error: *** JOB 18599512 ON n1720 CANCELLED AT 2025-07-24T00:51:19 DUE TO TIME LIMIT ***



⚠ Job 18599509 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599509, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_submitted.pkl
slurmstepd: error: *** JOB 18599509 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599522 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599522, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_submitted.pkl
slurmstepd: error: *** JOB 18599522 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***



⚠ Job 18599520 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,523) - Starting with JobEnvironment(job_id=18599520, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,525) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_submitted.pkl
slurmstepd: error: *** JOB 18599520 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***



⚠ Job 18599523 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599523, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_submitted.pkl
slurmstepd: error: *** JOB 18599523 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***



⚠ Job 18599517 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599517, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_submitted.pkl
slurmstepd: error: *** JOB 18599517 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***



⚠ Job 18599551 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,679) - Starting with JobEnvironment(job_id=18599551, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,683) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_submitted.pkl
slurmstepd: error: *** JOB 18599551 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599555 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,560) - Starting with JobEnvironment(job_id=18599555, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,563) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_submitted.pkl
slurmstepd: error: *** JOB 18599555 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599559 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,521) - Starting with JobEnvironment(job_id=18599559, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,523) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_submitted.pkl
slurmstepd: error: *** JOB 18599559 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599547 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:10,772) - Starting with JobEnvironment(job_id=18599547, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:10,774) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_submitted.pkl
slurmstepd: error: *** JOB 18599547 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***



⚠ Job 18599560 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,467) - Starting with JobEnvironment(job_id=18599560, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,472) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_submitted.pkl
slurmstepd: error: *** JOB 18599560 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599553 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,645) - Starting with JobEnvironment(job_id=18599553, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,649) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_submitted.pkl
slurmstepd: error: *** JOB 18599553 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599558 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,520) - Starting with JobEnvironment(job_id=18599558, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,523) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_submitted.pkl
slurmstepd: error: *** JOB 18599558 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599556 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,471) - Starting with JobEnvironment(job_id=18599556, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,473) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_submitted.pkl
slurmstepd: error: *** JOB 18599556 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599561 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:08:57,938) - Starting with JobEnvironment(job_id=18599561, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:08:57,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_submitted.pkl
slurmstepd: error: *** JOB 18599561 ON n1720 CANCELLED AT 2025-07-24T01:09:18 DUE TO TIME LIMIT ***



⚠ Job 18599554 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,646) - Starting with JobEnvironment(job_id=18599554, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_submitted.pkl
slurmstepd: error: *** JOB 18599554 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599549 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:10,772) - Starting with JobEnvironment(job_id=18599549, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:10,775) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_submitted.pkl
slurmstepd: error: *** JOB 18599549 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***



⚠ Job 18599552 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,687) - Starting with JobEnvironment(job_id=18599552, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,689) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_submitted.pkl
slurmstepd: error: *** JOB 18599552 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***



⚠ Job 18599562 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:08:57,938) - Starting with JobEnvironment(job_id=18599562, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:08:57,952) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_submitted.pkl
slurmstepd: error: *** JOB 18599562 ON n1720 CANCELLED AT 2025-07-24T01:09:18 DUE TO TIME LIMIT ***



⚠ Job 18599550 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:15,761) - Starting with JobEnvironment(job_id=18599550, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:15,774) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_submitted.pkl
slurmstepd: error: *** JOB 18599550 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***



⚠ Job 18599781 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599781, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_submitted.pkl
slurmstepd: error: *** JOB 18599781 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599787 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599787, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_submitted.pkl
slurmstepd: error: *** JOB 18599787 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599792 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599792, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_submitted.pkl
slurmstepd: error: *** JOB 18599792 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599789 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599789, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_submitted.pkl
slurmstepd: error: *** JOB 18599789 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599794 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599794, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,343) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_submitted.pkl
slurmstepd: error: *** JOB 18599794 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599777 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599777, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_submitted.pkl
slurmstepd: error: *** JOB 18599777 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599775 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599775, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_submitted.pkl
slurmstepd: error: *** JOB 18599775 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599791 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599791, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_submitted.pkl
slurmstepd: error: *** JOB 18599791 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599779 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599779, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_submitted.pkl
slurmstepd: error: *** JOB 18599779 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599806 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599806, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_submitted.pkl
slurmstepd: error: *** JOB 18599806 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599783 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599783, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_submitted.pkl
slurmstepd: error: *** JOB 18599783 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599786 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599786, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_submitted.pkl
slurmstepd: error: *** JOB 18599786 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599805 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,735) - Starting with JobEnvironment(job_id=18599805, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_submitted.pkl
slurmstepd: error: *** JOB 18599805 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599776 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599776, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_submitted.pkl
slurmstepd: error: *** JOB 18599776 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599798 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599798, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,343) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_submitted.pkl
slurmstepd: error: *** JOB 18599798 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599797 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599797, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_submitted.pkl
slurmstepd: error: *** JOB 18599797 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599784 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599784, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_submitted.pkl
slurmstepd: error: *** JOB 18599784 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599790 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599790, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_submitted.pkl
slurmstepd: error: *** JOB 18599790 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599804 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599804, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_submitted.pkl
slurmstepd: error: *** JOB 18599804 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599802 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599802, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_submitted.pkl
slurmstepd: error: *** JOB 18599802 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599782 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599782, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,557) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_submitted.pkl
slurmstepd: error: *** JOB 18599782 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599788 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599788, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,605) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_submitted.pkl
slurmstepd: error: *** JOB 18599788 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599795 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599795, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_submitted.pkl
slurmstepd: error: *** JOB 18599795 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599803 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599803, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_submitted.pkl
slurmstepd: error: *** JOB 18599803 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599780 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599780, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_submitted.pkl
slurmstepd: error: *** JOB 18599780 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599801 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,735) - Starting with JobEnvironment(job_id=18599801, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,748) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_submitted.pkl
slurmstepd: error: *** JOB 18599801 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599799 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599799, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_submitted.pkl
slurmstepd: error: *** JOB 18599799 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599800 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599800, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_submitted.pkl
slurmstepd: error: *** JOB 18599800 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599778 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,555) - Starting with JobEnvironment(job_id=18599778, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_submitted.pkl
slurmstepd: error: *** JOB 18599778 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***



⚠ Job 18599793 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599793, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_submitted.pkl
slurmstepd: error: *** JOB 18599793 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599785 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599785, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_submitted.pkl
slurmstepd: error: *** JOB 18599785 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***



⚠ Job 18599796 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599796, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_submitted.pkl
slurmstepd: error: *** JOB 18599796 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***



⚠ Job 18599807 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599807, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_submitted.pkl
slurmstepd: error: *** JOB 18599807 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***



⚠ Job 18599808 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599808, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,632) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_submitted.pkl
slurmstepd: error: *** JOB 18599808 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***



⚠ Job 18599809 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599809, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_submitted.pkl
slurmstepd: error: *** JOB 18599809 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***



⚠ Job 18599814 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599814, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,966) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_submitted.pkl
slurmstepd: error: *** JOB 18599814 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599810 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599810, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_submitted.pkl
slurmstepd: error: *** JOB 18599810 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***



⚠ Job 18599815 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,956) - Starting with JobEnvironment(job_id=18599815, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,963) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_submitted.pkl
slurmstepd: error: *** JOB 18599815 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599818 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599818, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,970) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_submitted.pkl
slurmstepd: error: *** JOB 18599818 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599812 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599812, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_submitted.pkl
slurmstepd: error: *** JOB 18599812 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***



⚠ Job 18599822 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599822, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_submitted.pkl
slurmstepd: error: *** JOB 18599822 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599823 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:35:42,717) - Starting with JobEnvironment(job_id=18599823, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:35:42,721) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_submitted.pkl
slurmstepd: error: *** JOB 18599823 ON n1720 CANCELLED AT 2025-07-24T03:35:56 DUE TO TIME LIMIT ***



⚠ Job 18599819 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599819, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_submitted.pkl
slurmstepd: error: *** JOB 18599819 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599816 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,956) - Starting with JobEnvironment(job_id=18599816, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_submitted.pkl
slurmstepd: error: *** JOB 18599816 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599811 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599811, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_submitted.pkl
slurmstepd: error: *** JOB 18599811 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***



⚠ Job 18599817 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,953) - Starting with JobEnvironment(job_id=18599817, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_submitted.pkl
slurmstepd: error: *** JOB 18599817 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599824 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:35:42,717) - Starting with JobEnvironment(job_id=18599824, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:35:42,727) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_submitted.pkl
slurmstepd: error: *** JOB 18599824 ON n1720 CANCELLED AT 2025-07-24T03:35:56 DUE TO TIME LIMIT ***



⚠ Job 18600073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,598) - Starting with JobEnvironment(job_id=18600073, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_submitted.pkl
slurmstepd: error: *** JOB 18600073 ON n1720 CANCELLED AT 2025-07-24T06:21:14 DUE TO TIME LIMIT ***



⚠ Job 18600072 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600072, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_submitted.pkl
slurmstepd: error: *** JOB 18600072 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***



⚠ Job 18600074 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,601) - Starting with JobEnvironment(job_id=18600074, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,603) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_submitted.pkl
slurmstepd: error: *** JOB 18600074 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600068, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_submitted.pkl
slurmstepd: error: *** JOB 18600068 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***



⚠ Job 18600081 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,586) - Starting with JobEnvironment(job_id=18600081, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,588) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_submitted.pkl
slurmstepd: error: *** JOB 18600081 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600070 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,982) - Starting with JobEnvironment(job_id=18600070, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,984) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_submitted.pkl
slurmstepd: error: *** JOB 18600070 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***



⚠ Job 18600067 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600067, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_submitted.pkl
slurmstepd: error: *** JOB 18600067 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***



⚠ Job 18600076 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,302) - Starting with JobEnvironment(job_id=18600076, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,305) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_submitted.pkl
slurmstepd: error: *** JOB 18600076 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600080 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,320) - Starting with JobEnvironment(job_id=18600080, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,323) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_submitted.pkl
slurmstepd: error: *** JOB 18600080 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,303) - Starting with JobEnvironment(job_id=18600075, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,305) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_submitted.pkl
slurmstepd: error: *** JOB 18600075 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600082 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,324) - Starting with JobEnvironment(job_id=18600082, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,325) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_submitted.pkl
slurmstepd: error: *** JOB 18600082 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600078 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,338) - Starting with JobEnvironment(job_id=18600078, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,340) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_submitted.pkl
slurmstepd: error: *** JOB 18600078 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600077 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,574) - Starting with JobEnvironment(job_id=18600077, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,578) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_submitted.pkl
slurmstepd: error: *** JOB 18600077 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:15,183) - Starting with JobEnvironment(job_id=18600079, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:15,190) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_submitted.pkl
slurmstepd: error: *** JOB 18600079 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:15:57,782) - Starting with JobEnvironment(job_id=18600066, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:15:57,791) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_submitted.pkl
slurmstepd: error: *** JOB 18600066 ON n1720 CANCELLED AT 2025-07-24T06:16:09 DUE TO TIME LIMIT ***



⚠ Job 18600083 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:25:59,608) - Starting with JobEnvironment(job_id=18600083, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:25:59,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_submitted.pkl
slurmstepd: error: *** JOB 18600083 ON n1720 CANCELLED AT 2025-07-24T06:26:14 DUE TO TIME LIMIT ***



⚠ Job 18600084 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:25:59,608) - Starting with JobEnvironment(job_id=18600084, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:25:59,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_submitted.pkl
slurmstepd: error: *** JOB 18600084 ON n1720 CANCELLED AT 2025-07-24T06:26:15 DUE TO TIME LIMIT ***



⚠ Job 18600252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_submitted.pkl
slurmstepd: error: *** JOB 18600252 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***



⚠ Job 18600241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600241, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_submitted.pkl
slurmstepd: error: *** JOB 18600241 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_submitted.pkl
slurmstepd: error: *** JOB 18600251 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***



⚠ Job 18600240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600240, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_submitted.pkl
slurmstepd: error: *** JOB 18600240 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600238 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600238, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_submitted.pkl
slurmstepd: error: *** JOB 18600238 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_submitted.pkl
slurmstepd: error: *** JOB 18600247 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600263 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600263, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_submitted.pkl
slurmstepd: error: *** JOB 18600263 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600239, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_submitted.pkl
slurmstepd: error: *** JOB 18600239 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600243, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_submitted.pkl
slurmstepd: error: *** JOB 18600243 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_submitted.pkl
slurmstepd: error: *** JOB 18600250 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***



⚠ Job 18600262 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,789) - Starting with JobEnvironment(job_id=18600262, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_submitted.pkl
slurmstepd: error: *** JOB 18600262 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,062) - Starting with JobEnvironment(job_id=18600248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,065) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_submitted.pkl
slurmstepd: error: *** JOB 18600248 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***



⚠ Job 18600267 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,696) - Starting with JobEnvironment(job_id=18600267, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,698) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_submitted.pkl
slurmstepd: error: *** JOB 18600267 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600268 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,696) - Starting with JobEnvironment(job_id=18600268, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,698) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_submitted.pkl
slurmstepd: error: *** JOB 18600268 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,063) - Starting with JobEnvironment(job_id=18600249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,065) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_submitted.pkl
slurmstepd: error: *** JOB 18600249 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***



⚠ Job 18600244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600244, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_submitted.pkl
slurmstepd: error: *** JOB 18600244 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600255 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:48:32,377) - Starting with JobEnvironment(job_id=18600255, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:48:32,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_submitted.pkl
slurmstepd: error: *** JOB 18600255 ON n1720 CANCELLED AT 2025-07-24T08:48:57 DUE TO TIME LIMIT ***



⚠ Job 18600261 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600261, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_submitted.pkl
slurmstepd: error: *** JOB 18600261 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600254 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:48:32,377) - Starting with JobEnvironment(job_id=18600254, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:48:32,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_submitted.pkl
slurmstepd: error: *** JOB 18600254 ON n1720 CANCELLED AT 2025-07-24T08:48:57 DUE TO TIME LIMIT ***



⚠ Job 18600258 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:09,538) - Starting with JobEnvironment(job_id=18600258, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:09,541) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_submitted.pkl
slurmstepd: error: *** JOB 18600258 ON n1720 CANCELLED AT 2025-07-24T08:51:29 DUE TO TIME LIMIT ***



⚠ Job 18600265 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600265, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_submitted.pkl
slurmstepd: error: *** JOB 18600265 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600260 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600260, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_submitted.pkl
slurmstepd: error: *** JOB 18600260 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600266 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,689) - Starting with JobEnvironment(job_id=18600266, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,697) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_submitted.pkl
slurmstepd: error: *** JOB 18600266 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_submitted.pkl
slurmstepd: error: *** JOB 18600246 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600259 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,698) - Starting with JobEnvironment(job_id=18600259, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,699) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_submitted.pkl
slurmstepd: error: *** JOB 18600259 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600264 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600264, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_submitted.pkl
slurmstepd: error: *** JOB 18600264 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600242, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_submitted.pkl
slurmstepd: error: *** JOB 18600242 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600245, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_submitted.pkl
slurmstepd: error: *** JOB 18600245 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***



⚠ Job 18600237 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:22,397) - Starting with JobEnvironment(job_id=18600237, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:22,400) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_submitted.pkl
slurmstepd: error: *** JOB 18600237 ON n1720 CANCELLED AT 2025-07-24T08:44:27 DUE TO TIME LIMIT ***



⚠ Job 18600970 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:26,743) - Starting with JobEnvironment(job_id=18600970, hostname=n1268, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:26,746) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_submitted.pkl
slurmstepd: error: *** JOB 18600970 ON n1268 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600975 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600975, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_submitted.pkl
slurmstepd: error: *** JOB 18600975 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600972 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,157) - Starting with JobEnvironment(job_id=18600972, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_submitted.pkl
slurmstepd: error: *** JOB 18600972 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600976 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,598) - Starting with JobEnvironment(job_id=18600976, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_submitted.pkl
slurmstepd: error: *** JOB 18600976 ON n1178 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600967 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,216) - Starting with JobEnvironment(job_id=18600967, hostname=n1270, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,219) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_submitted.pkl
slurmstepd: error: *** JOB 18600967 ON n1270 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600968 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,216) - Starting with JobEnvironment(job_id=18600968, hostname=n1270, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,218) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_submitted.pkl
slurmstepd: error: *** JOB 18600968 ON n1270 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600969 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:26,744) - Starting with JobEnvironment(job_id=18600969, hostname=n1268, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:26,746) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_submitted.pkl
slurmstepd: error: *** JOB 18600969 ON n1268 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600974 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600974, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,159) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_submitted.pkl
slurmstepd: error: *** JOB 18600974 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600971 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600971, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_submitted.pkl
slurmstepd: error: *** JOB 18600971 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600973 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600973, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,159) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_submitted.pkl
slurmstepd: error: *** JOB 18600973 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***



⚠ Job 18600980 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600980, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_submitted.pkl
slurmstepd: error: *** JOB 18600980 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18600981 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600981, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_submitted.pkl
slurmstepd: error: *** JOB 18600981 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18600982 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600982, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_submitted.pkl
slurmstepd: error: *** JOB 18600982 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18600978 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600978, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,796) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_submitted.pkl
slurmstepd: error: *** JOB 18600978 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18600977 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600977, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_submitted.pkl
slurmstepd: error: *** JOB 18600977 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18600979 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600979, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,796) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_submitted.pkl
slurmstepd: error: *** JOB 18600979 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***



⚠ Job 18601071 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601071, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_submitted.pkl
slurmstepd: error: *** JOB 18601071 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***



⚠ Job 18601052 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,993) - Starting with JobEnvironment(job_id=18601052, hostname=n1626, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_submitted.pkl
slurmstepd: error: *** JOB 18601052 ON n1626 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601067 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601067, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_submitted.pkl
slurmstepd: error: *** JOB 18601067 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***



⚠ Job 18601065 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601065, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_submitted.pkl
slurmstepd: error: *** JOB 18601065 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***



⚠ Job 18601068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601068, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_submitted.pkl
slurmstepd: error: *** JOB 18601068 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***



⚠ Job 18601075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601075, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_submitted.pkl
slurmstepd: error: *** JOB 18601075 ON n1255 CANCELLED AT 2025-07-24T11:56:21 DUE TO TIME LIMIT ***



⚠ Job 18601050 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,290) - Starting with JobEnvironment(job_id=18601050, hostname=n1513, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,293) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_submitted.pkl
slurmstepd: error: *** JOB 18601050 ON n1513 CANCELLED AT 2025-07-24T11:53:20 DUE TO TIME LIMIT ***



⚠ Job 18601060 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:17,966) - Starting with JobEnvironment(job_id=18601060, hostname=n1347, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:17,968) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_submitted.pkl
slurmstepd: error: *** JOB 18601060 ON n1347 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601066, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_submitted.pkl
slurmstepd: error: *** JOB 18601066 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***



⚠ Job 18601061 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,123) - Starting with JobEnvironment(job_id=18601061, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,125) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_submitted.pkl
slurmstepd: error: *** JOB 18601061 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601055 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,457) - Starting with JobEnvironment(job_id=18601055, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,459) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_submitted.pkl
slurmstepd: error: *** JOB 18601055 ON n1588 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601059 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:17,965) - Starting with JobEnvironment(job_id=18601059, hostname=n1347, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:17,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_submitted.pkl
slurmstepd: error: *** JOB 18601059 ON n1347 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601063 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,124) - Starting with JobEnvironment(job_id=18601063, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,126) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_submitted.pkl
slurmstepd: error: *** JOB 18601063 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601062 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,124) - Starting with JobEnvironment(job_id=18601062, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,126) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_submitted.pkl
slurmstepd: error: *** JOB 18601062 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601064 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,123) - Starting with JobEnvironment(job_id=18601064, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,125) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_submitted.pkl
slurmstepd: error: *** JOB 18601064 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601096 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601096, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_submitted.pkl
slurmstepd: error: *** JOB 18601096 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601105 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601105, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_submitted.pkl
slurmstepd: error: *** JOB 18601105 ON n1255 CANCELLED AT 2025-07-24T12:02:52 DUE TO TIME LIMIT ***



⚠ Job 18601079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:13,935) - Starting with JobEnvironment(job_id=18601079, hostname=n1490, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:13,937) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_submitted.pkl
slurmstepd: error: *** JOB 18601079 ON n1490 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601102 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601102, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,057) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_submitted.pkl
slurmstepd: error: *** JOB 18601102 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601104 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601104, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_submitted.pkl
slurmstepd: error: *** JOB 18601104 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601092 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601092, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_submitted.pkl
slurmstepd: error: *** JOB 18601092 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601106 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601106, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_submitted.pkl
slurmstepd: error: *** JOB 18601106 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601091 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601091, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_submitted.pkl
slurmstepd: error: *** JOB 18601091 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601090 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601090, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_submitted.pkl
slurmstepd: error: *** JOB 18601090 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***



⚠ Job 18601111 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601111, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_submitted.pkl
slurmstepd: error: *** JOB 18601111 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***



⚠ Job 18601109 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601109, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_submitted.pkl
slurmstepd: error: *** JOB 18601109 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***



⚠ Job 18601110 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601110, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_submitted.pkl
slurmstepd: error: *** JOB 18601110 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***



⚠ Job 18601121 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601121, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_submitted.pkl
slurmstepd: error: *** JOB 18601121 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***



⚠ Job 18601107 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,988) - Starting with JobEnvironment(job_id=18601107, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,990) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_submitted.pkl
slurmstepd: error: *** JOB 18601107 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***

Copy raw data to clipboard

Download »oo_errors.txt« as file

Copy raw data to clipboard

Download »outfile« as file

To cancel, press CTRL c, then run 'scancel 18592137'
⠋ Importing logging...
⠋ Importing warnings...
⠋ Importing argparse...
⠋ Importing datetime...
⠋ Importing dataclass...
⠋ Importing hashlib...
⠋ Importing socket...
⠋ Importing stat...
⠋ Importing pwd...
⠋ Importing signal...
⠋ Importing base64...
⠋ Importing json...
⠋ Importing yaml...
⠋ Importing toml...
⠋ Importing csv...
⠋ Importing ast...
⠋ Importing rich.table...
⠋ Importing rich print...
⠋ Importing rich.pretty...
⠋ Importing rich.prompt...
⠋ Importing types.FunctionType...
⠋ Importing typing...
⠋ Importing ThreadPoolExecutor...
⠋ Importing submitit.LocalExecutor...
⠋ Importing submitit.Job...
⠋ Importing importlib.util...
⠋ Importing inspect...
⠋ Importing platform...
⠋ Importing inspect frame info...
⠋ Importing pathlib.Path...
⠋ Importing uuid...
⠋ Importing traceback...
⠋ Importing cowsay...
⠋ Importing psutil...
⠋ Importing shutil...
⠋ Importing itertools.combinations...
⠋ Importing os.listdir...
⠋ Importing os.path...
⠋ Importing PIL.Image...
⠋ Importing sixel...
⠋ Importing subprocess...
⠋ Importing tqdm...
⠼ Importing beartype...
⠋ Importing statistics...
⠋ Trying to import pyfiglet...
⠸ Importing helpers...
⠋ Parsing arguments...
⠧ Importing torch...
⠋ Importing numpy...
⠋ Importing collections...
⠙ Importing ax...
⠋ Importing ax.core.generator_run...
⠋ Importing Cont_X_trans and Y_trans from ax.modelbridge.registry...
⠋ Importing ax.core.arm...
⠋ Importing ax.core.objective...
⠋ Importing ax.core.Metric...
⠋ Importing ax.exceptions.core...
⠋ Importing ax.exceptions.generation_strategy...
⠋ Importing CORE_DECODER_REGISTRY...
⠋ Trying ax.generation_strategy.generation_node...
⠋ Importing GenerationStep, GenerationStrategy from generation_strategy...
⠋ Importing GenerationNode from generation_node...
⠋ Importing ExternalGenerationNode...
⠋ Importing MaxTrials...
⠋ Importing GeneratorSpec...
⠋ Importing Models from ax.modelbridge.registry...
⠋ Importing get_pending_observation_features...
⠋ Importing load_experiment...
⠋ Importing save_experiment...
⠋ Importing save_experiment_to_db...
⠋ Importing TrialStatus...
⠋ Importing Data...
⠋ Importing Experiment...
⠋ Importing parameter types...
⠋ Importing TParameterization...
⠋ Importing pandas...
⠋ Importing AxClient and ObjectiveProperties...
⠋ Importing RandomForestRegressor...
⠋ Importing botorch...
⠋ Importing submitit...
⠋ Importing ax logger...
⠋ Importing SQL-Storage-Stuff...
Run-UUID: b3597690-010b-4447-b198-5e13545a2c60
  __                         __           __ 
 /\_\/                   o  /\_\/        /  )
|    | _  _  _    _  _     |    |  _ _|_   / 
|    |/ |/ |/ |  / |/ |  | |    ||/ \_|   /  
 \__/   |  |  |_/  |  |_/|_/\__/ |__/ |_//___
                                /|           
                                \|           

⠋ Writing worker creation log...
omniopt --partition=barnard --experiment_name=mnist_cpu --mem_gb=5 --time=1440 --worker_timeout=60 --max_eval=500 --num_parallel_jobs=50 --gpus=0 --num_random_steps=20 --follow --live_share --send_anonymized_usage_stats --result_names VAL_ACC=max --run_program=cHl0aG9uMyAudGVzdHMvbW5pc3QvdHJhaW4gLS1lcG9jaHMgJWVwb2NocyAtLWxlYXJuaW5nX3JhdGUgJWxyIC0tYmF0Y2hfc2l6ZSAlYmF0Y2hfc2l6ZSAtLWhpZGRlbl9zaXplICVoaWRkZW5fc2l6ZSAtLWRyb3BvdXQgJWRyb3BvdXQgLS1hY3RpdmF0aW9uICVhY3RpdmF0aW9uIC0tbnVtX2RlbnNlX2xheWVycyAlbnVtX2RlbnNlX2xheWVycyAtLWluaXQgJWluaXQgLS13ZWlnaHRfZGVjYXkgJXdlaWdodF9kZWNheQ== --cpus_per_task=1 --nodes_per_job=1 --revert_to_random_when_seemingly_exhausted --model=BOTORCH_MODULAR --n_estimators_randomforest=100 --run_mode=local --occ_type=euclid --main_process_gb=8 --max_nr_of_zero_results=50 --slurm_signal_delay_s=0 --max_failed_jobs=0 --max_attempts_for_generation=20 --num_restarts=20 --raw_samples=1024 --max_abandoned_retrial=20 --max_num_of_parallel_sruns=16 --generate_all_jobs_at_once --parameter epochs range 10 200 int false --parameter lr range 0.0001 0.1 float false --parameter batch_size range 8 4096 int false --parameter hidden_size range 8 8192 int false --parameter dropout range 0 0.5 float false --parameter activation fixed leaky_relu --parameter num_dense_layers range 1 4 int false --parameter init fixed normal --parameter weight_decay range 0 1 float false --ui_url aHR0cHM6Ly9pbWFnZXNlZy5zY2Fkcy5kZS9vbW5pYXgvZ3VpP3BhcnRpdGlvbj1iYXJuYXJkJmV4cGVyaW1lbnRfbmFtZT1tbmlzdF9jcHUmcmVzZXJ2YXRpb249JmFjY291bnQ9Jm1lbV9nYj01JnRpbWU9MjQwMCZ3b3JrZXJfdGltZW91dD02MCZtYXhfZXZhbD01MDAmbnVtX3BhcmFsbGVsX2pvYnM9NTAmZ3B1cz0wJm51bV9yYW5kb21fc3RlcHM9MjAmZm9sbG93PTEmbGl2ZV9zaGFyZT0xJnNlbmRfYW5vbnltaXplZF91c2FnZV9zdGF0cz0xJmNvbnN0cmFpbnRzPSZyZXN1bHRfbmFtZXM9VkFMX0FDQyUzRG1heCZydW5fcHJvZ3JhbT1weXRob24zJTIwLnRlc3RzJTJGbW5pc3QlMkZ0cmFpbiUyMC0tZXBvY2hzJTIwJTI1ZXBvY2hzJTIwLS1sZWFybmluZ19yYXRlJTIwJTI1bHIlMjAtLWJhdGNoX3NpemUlMjAlMjViYXRjaF9zaXplJTIwLS1oaWRkZW5fc2l6ZSUyMCUyNWhpZGRlbl9zaXplJTIwLS1kcm9wb3V0JTIwJTI1ZHJvcG91dCUyMC0tYWN0aXZhdGlvbiUyMCUyNWFjdGl2YXRpb24lMjAtLW51bV9kZW5zZV9sYXllcnMlMjAlMjVudW1fZGVuc2VfbGF5ZXJzJTIwLS1pbml0JTIwJTI1aW5pdCUyMC0td2VpZ2h0X2RlY2F5JTIwJTI1d2VpZ2h0X2RlY2F5JmNwdXNfcGVyX3Rhc2s9MSZub2Rlc19wZXJfam9iPTEmc2VlZD0mZHJ5cnVuPTAmZGVidWc9MCZyZXZlcnRfdG9fcmFuZG9tX3doZW5fc2VlbWluZ2x5X2V4aGF1c3RlZD0xJmdyaWRzZWFyY2g9MCZtb2RlbD1CT1RPUkNIX01PRFVMQVImZXh0ZXJuYWxfZ2VuZXJhdG9yPSZuX2VzdGltYXRvcnNfcmFuZG9tZm9yZXN0PTEwMCZpbnN0YWxsYXRpb25fbWV0aG9kPWNsb25lJnJ1bl9tb2RlPWxvY2FsJmRpc2FibGVfdHFkbT0wJnZlcmJvc2VfdHFkbT0wJmZvcmNlX2xvY2FsX2V4ZWN1dGlvbj0wJmF1dG9fZXhjbHVkZV9kZWZlY3RpdmVfaG9zdHM9MCZzaG93X3NpeGVsX2dlbmVyYWw9MCZzaG93X3NpeGVsX3RyaWFsX2luZGV4X3Jlc3VsdD0wJnNob3dfc2l4ZWxfc2NhdHRlcj0wJnNob3dfd29ya2VyX3BlcmNlbnRhZ2VfdGFibGVfYXRfZW5kPTAmb2NjPTAmb2NjX3R5cGU9ZXVjbGlkJm5vX3NsZWVwPTAmc2x1cm1fdXNlX3NydW49MCZ2ZXJib3NlX2JyZWFrX3J1bl9zZWFyY2hfdGFibGU9MCZhYmJyZXZpYXRlX2pvYl9uYW1lcz0wJm1haW5fcHJvY2Vzc19nYj04Jm1heF9ucl9vZl96ZXJvX3Jlc3VsdHM9NTAmc2x1cm1fc2lnbmFsX2RlbGF5X3M9MCZtYXhfZmFpbGVkX2pvYnM9MCZleGNsdWRlPSZ1c2VybmFtZT0mZ2VuZXJhdGlvbl9zdHJhdGVneT0mcm9vdF92ZW52X2Rpcj0md29ya2Rpcj0mZG9udF9qaXRfY29tcGlsZT0wJmZpdF9vdXRfb2ZfZGVzaWduPTAmcmVmaXRfb25fY3Y9MCZzaG93X2dlbmVyYXRlX3RpbWVfdGFibGU9MCZkb250X3dhcm1fc3RhcnRfcmVmaXR0aW5nPTAmbWF4X2F0dGVtcHRzX2Zvcl9nZW5lcmF0aW9uPTIwJm51bV9yZXN0YXJ0cz0yMCZyYXdfc2FtcGxlcz0xMDI0Jm1heF9hYmFuZG9uZWRfcmV0cmlhbD0yMCZtYXhfbnVtX29mX3BhcmFsbGVsX3NydW5zPTE2JmZvcmNlX2Nob2ljZV9mb3JfcmFuZ2VzPTAmbm9fdHJhbnNmb3JtX2lucHV0cz0wJmZpdF9hYmFuZG9uZWQ9MCZub19ub3JtYWxpemVfeT0wJnZlcmJvc2U9MCZnZW5lcmF0ZV9hbGxfam9ic19hdF9vbmNlPTEmZmxhbWVfZ3JhcGg9MCZjaGVja291dF90b19sYXRlc3RfdGVzdGVkX3ZlcnNpb249MCZwYXJhbWV0ZXJfMF9uYW1lPWVwb2NocyZwYXJhbWV0ZXJfMF90eXBlPXJhbmdlJnBhcmFtZXRlcl8wX21pbj0xMCZwYXJhbWV0ZXJfMF9tYXg9MjAwJnBhcmFtZXRlcl8wX251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfMF9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzFfbmFtZT1sciZwYXJhbWV0ZXJfMV90eXBlPXJhbmdlJnBhcmFtZXRlcl8xX21pbj0wLjAwMDEmcGFyYW1ldGVyXzFfbWF4PTAuMSZwYXJhbWV0ZXJfMV9udW1iZXJfdHlwZT1mbG9hdCZwYXJhbWV0ZXJfMV9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzJfbmFtZT1iYXRjaF9zaXplJnBhcmFtZXRlcl8yX3R5cGU9cmFuZ2UmcGFyYW1ldGVyXzJfbWluPTgmcGFyYW1ldGVyXzJfbWF4PTQwOTYmcGFyYW1ldGVyXzJfbnVtYmVyX3R5cGU9aW50JnBhcmFtZXRlcl8yX2xvZ19zY2FsZT1mYWxzZSZwYXJhbWV0ZXJfM19uYW1lPWhpZGRlbl9zaXplJnBhcmFtZXRlcl8zX3R5cGU9cmFuZ2UmcGFyYW1ldGVyXzNfbWluPTgmcGFyYW1ldGVyXzNfbWF4PTgxOTImcGFyYW1ldGVyXzNfbnVtYmVyX3R5cGU9aW50JnBhcmFtZXRlcl8zX2xvZ19zY2FsZT1mYWxzZSZwYXJhbWV0ZXJfNF9uYW1lPWRyb3BvdXQmcGFyYW1ldGVyXzRfdHlwZT1yYW5nZSZwYXJhbWV0ZXJfNF9taW49MCZwYXJhbWV0ZXJfNF9tYXg9MC41JnBhcmFtZXRlcl80X251bWJlcl90eXBlPWZsb2F0JnBhcmFtZXRlcl80X2xvZ19zY2FsZT1mYWxzZSZwYXJhbWV0ZXJfNV9uYW1lPWFjdGl2YXRpb24mcGFyYW1ldGVyXzVfdHlwZT1maXhlZCZwYXJhbWV0ZXJfNV92YWx1ZT1sZWFreV9yZWx1JnBhcmFtZXRlcl82X25hbWU9bnVtX2RlbnNlX2xheWVycyZwYXJhbWV0ZXJfNl90eXBlPXJhbmdlJnBhcmFtZXRlcl82X21pbj0xJnBhcmFtZXRlcl82X21heD00JnBhcmFtZXRlcl82X251bWJlcl90eXBlPWludCZwYXJhbWV0ZXJfNl9sb2dfc2NhbGU9ZmFsc2UmcGFyYW1ldGVyXzdfbmFtZT1pbml0JnBhcmFtZXRlcl83X3R5cGU9Zml4ZWQmcGFyYW1ldGVyXzdfdmFsdWU9bm9ybWFsJnBhcmFtZXRlcl84X25hbWU9d2VpZ2h0X2RlY2F5JnBhcmFtZXRlcl84X3R5cGU9cmFuZ2UmcGFyYW1ldGVyXzhfbWluPTAmcGFyYW1ldGVyXzhfbWF4PTEmcGFyYW1ldGVyXzhfbnVtYmVyX3R5cGU9ZmxvYXQmcGFyYW1ldGVyXzhfbG9nX3NjYWxlPWZhbHNlJnBhcnRpdGlvbj1iYXJuYXJkJm51bV9wYXJhbWV0ZXJzPTk=
⠋ Disabling logging...
⠋ Setting run folder...
⠋ Creating folder /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2...
⠋ Writing revert_to_random_when_seemingly_exhausted file ...
⠋ Writing username state file...
⠋ Writing result names file...
⠋ Writing result min/max file...
⠋ Saving state files...
Run-folder: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2
⠋ Printing run info...
⠋ Initializing NVIDIA-Logs...
⠋ Writing ui_url file if it is present...
⠋ Writing live_share file if it is present...
⠋ Writing job_start_time file...
⠹ Writing git info file...
⠋ Checking max_eval...
Warning: --num_random_steps 20 is smaller than --num_parallel_jobs 50. It's recommended that --num_parallel_jobs is the same as or a multiple of --num_random_steps
⠋ Calculating number of steps...
⠋ Adding excluded nodes...
⠋ Handling random steps...
⠋ Initializing ax_client...
[WARNING 07-23 13:32:13] ax.service.ax_client: Selecting a GenerationStrategy when using BatchTrials is in beta. Double check the recommended strategy matches your expectations.
⠋ Setting orchestrator...
You have 2 CPUs available for the main process. No CUDA devices found. Generation strategy: SOBOL for 20 steps and then BOTORCH_MODULAR for 480 steps.
Run-Program: python3 .tests/mnist/train --epochs %epochs --learning_rate %lr --batch_size %batch_size --hidden_size %hidden_size --dropout %dropout --activation %activation --num_dense_layers %num_dense_layers --init %init --weight_decay %weight_decay
                                  Experiment parameters                                   
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃ Name             ┃ Type  ┃ Lower bound ┃ Upper bound ┃ Values     ┃ Type  ┃ Log Scale? ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│ epochs           │ range │ 10          │ 200         │            │ int   │ No         │
│ lr               │ range │ 0.0001      │ 0.1         │            │ float │ No         │
│ batch_size       │ range │ 8           │ 4096        │            │ int   │ No         │
│ hidden_size      │ range │ 8           │ 8192        │            │ int   │ No         │
│ dropout          │ range │ 0           │ 0.5         │            │ float │ No         │
│ activation       │ fixed │             │             │ leaky_relu │       │            │
│ num_dense_layers │ range │ 1           │ 4           │            │ int   │ No         │
│ init             │ fixed │             │             │ normal     │       │            │
│ weight_decay     │ range │ 0           │ 1           │            │ float │ No         │
└──────────────────┴───────┴─────────────┴─────────────┴────────────┴───────┴────────────┘
        Result-Names         
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Result-Name ┃ Min or max? ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ VAL_ACC     │         max │
└─────────────┴─────────────┘
See https://imageseg.scads.de/omniax/share?user_id=pwinkler&experiment_name=mnist_cpu&run_nr=2 for live-results.

█▀▀▀▀▀█ ▄ █ ▀█▄▄█ █▀▀ ▄▀ █▄▄  █▀▀▀▀▀█
█ ███ █ ▄▄▄ █▄▄ █▄ █▀▀▄▄█▄█▀▄ █ ███ █
█ ▀▀▀ █ ▄█ ▄▄▄▀█▀▀▀▀█  ▀ ▄▀▄█ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀▄█▄▀ ▀ █ ▀▄█▄█▄▀ █▄▀ ▀▀▀▀▀▀▀
█▀██▀ ▀▀▀█▄▀▄█▄ ▄▀▄▀█ ▀▄ █▀▀▄▀ █ █▄█ 
▀ ▀▀▀█▀▄▀▄▀ ▄ ███   ▄█▀ █  █▀▄▀▀▀█▀▄▀
▀ ▀ █▄▀▄ ▀  ▀▀▄▀██▀▄█▄▀▄ ▀▀ ▄▀▀█▀▀█▄▀
▀█▄▄ ▄▀▄ ▄▀█▀▀ ▀   ▄▄██   ▄▀▄▄█▄█▄▀▀█
▄▀ █▀ ▀▄▀▄▀▀▄▄ ▀██▄█▄▄██▄▀█ ▄▀▀ ▀▀█▄▀
  ▄▀██▀▀ █▄ ▄█▀▀▄   █▄  ▀▀ ██▀██▀ ▀▀█
▄▄▄   ▀▀▄ █ ▀ ▀▀ █▄▄▄▄ ▄▀▀▀ ▄█▀▄▀███▀
▀▀▄▄█▄▀▄ █ █▀ ▄██▀▄ █▄▀ ▄ █▀█▄▀█   ▀█
▄▄██▄▀▀▄█▀█▀▄▄▄▀██▄█▄▄▀█▀█▄ ▄ ▀▄▀▀██▀
█ ▀▀▀ ▀▀█   ▄█▀█  ▀ ▀██▄ ▀█▀█▀ ▀▀  ▀█
▀ ▀ ▀ ▀▀▄▀█ ▀ ▀▀█▀ █▄ █▀ ▀█▄█▀▀▀██▀  
█▀▀▀▀▀█ ▀█▀█▀ ▄▀▀▀▀▄ █ ▄█ █▄█ ▀ █  ▀▀
█ ███ █ █ ▄▀▄▄▄▀▄▀ ▄█▄█▄ ▀█▀▀████▀▀▄▄
█ ▀▀▀ █ █ █ ▄█▀▀█ ▄ ██   ▀▄▄▄▄▄█  ▀▀█
▀▀▀▀▀▀▀ ▀ ▀ ▀ ▀ ▀     ▀ ▀▀▀   ▀▀ ▀▀▀▀
Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                :   3%|░░░░░░░░░░| 13/500 [1:13:09<27:27:13, 202.94s/it]
⚠ Job 18592310 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,847) - Starting with JobEnvironment(job_id=18592310, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,849) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592310/18592310_submitted.pkl
slurmstepd: error: *** JOB 18592310 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***


⚠ Job 18592366 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592366, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592366/18592366_submitted.pkl
slurmstepd: error: *** JOB 18592366 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***


⚠ Job 18592320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,243) - Starting with JobEnvironment(job_id=18592320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592320/18592320_submitted.pkl
slurmstepd: error: *** JOB 18592320 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***


⚠ Job 18592316 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,823) - Starting with JobEnvironment(job_id=18592316, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,824) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592316/18592316_submitted.pkl
slurmstepd: error: *** JOB 18592316 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***


⚠ Job 18592308 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,907) - Starting with JobEnvironment(job_id=18592308, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,909) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592308/18592308_submitted.pkl
slurmstepd: error: *** JOB 18592308 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***


⚠ Job 18592314 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,823) - Starting with JobEnvironment(job_id=18592314, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,825) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592314/18592314_submitted.pkl
slurmstepd: error: *** JOB 18592314 ON n1720 CANCELLED AT 2025-07-23T14:45:12 DUE TO TIME LIMIT ***


⚠ Job 18592312 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:04,206) - Starting with JobEnvironment(job_id=18592312, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:04,207) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592312/18592312_submitted.pkl
slurmstepd: error: *** JOB 18592312 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***


⚠ Job 18592327 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,384) - Starting with JobEnvironment(job_id=18592327, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,387) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592327/18592327_submitted.pkl
slurmstepd: error: *** JOB 18592327 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***


⚠ Job 18592336 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,153) - Starting with JobEnvironment(job_id=18592336, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,155) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592336/18592336_submitted.pkl
slurmstepd: error: *** JOB 18592336 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***


⚠ Job 18592351 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,224) - Starting with JobEnvironment(job_id=18592351, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,226) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592351/18592351_submitted.pkl
slurmstepd: error: *** JOB 18592351 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***


⚠ Job 18592353 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,226) - Starting with JobEnvironment(job_id=18592353, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,227) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592353/18592353_submitted.pkl
slurmstepd: error: *** JOB 18592353 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***


⚠ Job 18592361 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592361, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592361/18592361_submitted.pkl
slurmstepd: error: *** JOB 18592361 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***


⚠ Job 18592367 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592367, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592367/18592367_submitted.pkl
slurmstepd: error: *** JOB 18592367 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***


⚠ Job 18592354 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,224) - Starting with JobEnvironment(job_id=18592354, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,226) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592354/18592354_submitted.pkl
slurmstepd: error: *** JOB 18592354 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***


⚠ Job 18592331 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,112) - Starting with JobEnvironment(job_id=18592331, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,114) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592331/18592331_submitted.pkl
slurmstepd: error: *** JOB 18592331 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***


⚠ Job 18592339 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,085) - Starting with JobEnvironment(job_id=18592339, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592339/18592339_submitted.pkl
slurmstepd: error: *** JOB 18592339 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***


⚠ Job 18592317 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:03,846) - Starting with JobEnvironment(job_id=18592317, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:03,848) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592317/18592317_submitted.pkl
slurmstepd: error: *** JOB 18592317 ON n1720 CANCELLED AT 2025-07-23T14:45:11 DUE TO TIME LIMIT ***


⚠ Job 18592332 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:36,153) - Starting with JobEnvironment(job_id=18592332, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:36,156) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592332/18592332_submitted.pkl
slurmstepd: error: *** JOB 18592332 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***


⚠ Job 18592344 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,084) - Starting with JobEnvironment(job_id=18592344, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592344/18592344_submitted.pkl
slurmstepd: error: *** JOB 18592344 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***


⚠ Job 18592322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592322, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,246) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592322/18592322_submitted.pkl
slurmstepd: error: *** JOB 18592322 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***

⚠ Job 18592341 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,083) - Starting with JobEnvironment(job_id=18592341, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592341/18592341_submitted.pkl
slurmstepd: error: *** JOB 18592341 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***


⚠ Job 18592319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592319/18592319_submitted.pkl
slurmstepd: error: *** JOB 18592319 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***


⚠ Job 18592338 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,085) - Starting with JobEnvironment(job_id=18592338, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,086) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592338/18592338_submitted.pkl
slurmstepd: error: *** JOB 18592338 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***

⚠ Job 18592329 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,384) - Starting with JobEnvironment(job_id=18592329, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,387) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592329/18592329_submitted.pkl
slurmstepd: error: *** JOB 18592329 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***

⚠ Job 18592355 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:36,206) - Starting with JobEnvironment(job_id=18592355, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:36,208) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592355/18592355_submitted.pkl
slurmstepd: error: *** JOB 18592355 ON n1720 CANCELLED AT 2025-07-23T14:47:42 DUE TO TIME LIMIT ***


⚠ Job 18592334 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,381) - Starting with JobEnvironment(job_id=18592334, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,386) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592334/18592334_submitted.pkl
slurmstepd: error: *** JOB 18592334 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***

⚠ Job 18592330 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,383) - Starting with JobEnvironment(job_id=18592330, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,385) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592330/18592330_submitted.pkl
slurmstepd: error: *** JOB 18592330 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***


⚠ Job 18592318 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,244) - Starting with JobEnvironment(job_id=18592318, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,246) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592318/18592318_submitted.pkl
slurmstepd: error: *** JOB 18592318 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***


⚠ Job 18592323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:45:36,243) - Starting with JobEnvironment(job_id=18592323, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:45:36,245) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592323/18592323_submitted.pkl
slurmstepd: error: *** JOB 18592323 ON n1720 CANCELLED AT 2025-07-23T14:45:42 DUE TO TIME LIMIT ***


⚠ Job 18592363 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592363, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,273) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592363/18592363_submitted.pkl
slurmstepd: error: *** JOB 18592363 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***



⚠ Job 18592333 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:46:35,383) - Starting with JobEnvironment(job_id=18592333, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:46:35,385) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592333/18592333_submitted.pkl
slurmstepd: error: *** JOB 18592333 ON n1720 CANCELLED AT 2025-07-23T14:46:41 DUE TO TIME LIMIT ***





⚠ Job 18592340 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:47:06,084) - Starting with JobEnvironment(job_id=18592340, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:47:06,088) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592340/18592340_submitted.pkl
slurmstepd: error: *** JOB 18592340 ON n1720 CANCELLED AT 2025-07-23T14:47:11 DUE TO TIME LIMIT ***

Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         :   3%|░░░░░░░░░░| 13/500 [1:21:40<27:27:13, 202.94s/it]
⚠ Job 18592364 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592364, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,273) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592364/18592364_submitted.pkl
slurmstepd: error: *** JOB 18592364 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***

Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     :   3%|░░░░░░░░░░| 13/500 [1:22:46<27:27:13, 202.94s/it]
⚠ Job 18592370 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:49:07,852) - Starting with JobEnvironment(job_id=18592370, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:49:07,855) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592370/18592370_submitted.pkl
slurmstepd: error: *** JOB 18592370 ON n1720 CANCELLED AT 2025-07-23T14:49:11 DUE TO TIME LIMIT ***


⚠ Job 18592368 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,271) - Starting with JobEnvironment(job_id=18592368, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592368/18592368_submitted.pkl
slurmstepd: error: *** JOB 18592368 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***


⚠ Job 18592369 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:48:37,270) - Starting with JobEnvironment(job_id=18592369, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:48:37,272) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592369/18592369_submitted.pkl
slurmstepd: error: *** JOB 18592369 ON n1720 CANCELLED AT 2025-07-23T14:48:41 DUE TO TIME LIMIT ***


⚠ Job 18592375 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_0_log.out
----------------------------------------
submitit INFO (2025-07-23 13:49:38,508) - Starting with JobEnvironment(job_id=18592375, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 13:49:38,510) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18592375/18592375_submitted.pkl
slurmstepd: error: *** JOB 18592375 ON n1720 CANCELLED AT 2025-07-23T14:49:41 DUE TO TIME LIMIT ***

Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job :   3%|░░░░░░░░░░| 13/500 [1:52:23<27:27:13, 202.94s/it]
Exit-Code 1, this means: There was an error regarding the python-syntax. Is your python-version too old? If you are on a slurm-system, this may mean that the job cannot be started, this can happen when you need a GPU but have not set one, or your home is full. Or you are on a cluster that needs jobs to allocate GPUs, but you have not allocated a GPU. Check stdout. May also happen when the job was cancelled due to CTRL-c, or when your virtual environment is outdated. Try deleting it, especially when you are running on a tested version. It usually is in ~/.omniax_... It may also be that your home is full and you tried to run an sbatch job.
Sobol, failed: 37, best VAL_ACC: 21.99, running 35 = ∑35/50, waiting for 35 jobs                            :   6%|▒░░░░░░░░░| 28/500 [2:47:48<63:09:21, 481.70s/it]
⚠ Job 18594393 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594393, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,991) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594393/18594393_submitted.pkl
slurmstepd: error: *** JOB 18594393 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***


⚠ Job 18594395 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594395, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594395/18594395_submitted.pkl
slurmstepd: error: *** JOB 18594395 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***


⚠ Job 18594403 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,789) - Starting with JobEnvironment(job_id=18594403, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594403/18594403_submitted.pkl
slurmstepd: error: *** JOB 18594403 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***


⚠ Job 18594392 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594392, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594392/18594392_submitted.pkl
slurmstepd: error: *** JOB 18594392 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***


⚠ Job 18594400 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:06,990) - Starting with JobEnvironment(job_id=18594400, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:06,992) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594400/18594400_submitted.pkl
slurmstepd: error: *** JOB 18594400 ON n1719 CANCELLED AT 2025-07-23T16:19:15 DUE TO TIME LIMIT ***


⚠ Job 18594402 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594402, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594402/18594402_submitted.pkl
slurmstepd: error: *** JOB 18594402 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***


⚠ Job 18594401 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594401, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594401/18594401_submitted.pkl
slurmstepd: error: *** JOB 18594401 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***


⚠ Job 18594405 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594405, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,789) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594405/18594405_submitted.pkl
slurmstepd: error: *** JOB 18594405 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***


⚠ Job 18594404 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,788) - Starting with JobEnvironment(job_id=18594404, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,789) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594404/18594404_submitted.pkl
slurmstepd: error: *** JOB 18594404 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***


⚠ Job 18594406 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:19:37,789) - Starting with JobEnvironment(job_id=18594406, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:19:37,790) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594406/18594406_submitted.pkl
slurmstepd: error: *** JOB 18594406 ON n1719 CANCELLED AT 2025-07-23T16:19:45 DUE TO TIME LIMIT ***

Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            :   6%|▒░░░░░░░░░| 28/500 [2:49:41<63:09:21, 481.70s/it]
⚠ Job 18594483 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:46,624) - Starting with JobEnvironment(job_id=18594483, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:46,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594483/18594483_submitted.pkl
slurmstepd: error: *** JOB 18594483 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***


⚠ Job 18594485 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,258) - Starting with JobEnvironment(job_id=18594485, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,261) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594485/18594485_submitted.pkl
slurmstepd: error: *** JOB 18594485 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***


⚠ Job 18594491 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594491, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594491/18594491_submitted.pkl
slurmstepd: error: *** JOB 18594491 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***

⚠ Job 18594502 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,633) - Starting with JobEnvironment(job_id=18594502, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,635) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594502/18594502_submitted.pkl
slurmstepd: error: *** JOB 18594502 ON n1719 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***



⚠ Job 18594607 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,721) - Starting with JobEnvironment(job_id=18594607, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594607/18594607_submitted.pkl
slurmstepd: error: *** JOB 18594607 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594585 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594585, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594585/18594585_submitted.pkl
slurmstepd: error: *** JOB 18594585 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***


⚠ Job 18594487 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594487, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594487/18594487_submitted.pkl
slurmstepd: error: *** JOB 18594487 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***


⚠ Job 18594603 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,710) - Starting with JobEnvironment(job_id=18594603, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,712) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594603/18594603_submitted.pkl
slurmstepd: error: *** JOB 18594603 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594602 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594602, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,724) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594602/18594602_submitted.pkl
slurmstepd: error: *** JOB 18594602 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594574 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:14,428) - Starting with JobEnvironment(job_id=18594574, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:14,430) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594574/18594574_submitted.pkl
slurmstepd: error: *** JOB 18594574 ON n1719 CANCELLED AT 2025-07-23T16:24:16 DUE TO TIME LIMIT ***


⚠ Job 18594552 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:23:18,421) - Starting with JobEnvironment(job_id=18594552, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:23:18,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594552/18594552_submitted.pkl
slurmstepd: error: *** JOB 18594552 ON n1719 CANCELLED AT 2025-07-23T16:23:15 DUE TO TIME LIMIT ***

⚠ Job 18594506 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,757) - Starting with JobEnvironment(job_id=18594506, hostname=n1714, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594506/18594506_submitted.pkl
slurmstepd: error: *** JOB 18594506 ON n1714 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***

⚠ Job 18594588 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594588, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594588/18594588_submitted.pkl
slurmstepd: error: *** JOB 18594588 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***


⚠ Job 18594605 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594605, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594605/18594605_submitted.pkl
slurmstepd: error: *** JOB 18594605 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594608 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,720) - Starting with JobEnvironment(job_id=18594608, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,722) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594608/18594608_submitted.pkl
slurmstepd: error: *** JOB 18594608 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***



⚠ Job 18594490 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594490, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594490/18594490_submitted.pkl
slurmstepd: error: *** JOB 18594490 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***

⚠ Job 18594486 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,258) - Starting with JobEnvironment(job_id=18594486, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594486/18594486_submitted.pkl
slurmstepd: error: *** JOB 18594486 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***


⚠ Job 18594589 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594589, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,270) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594589/18594589_submitted.pkl
slurmstepd: error: *** JOB 18594589 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***




⚠ Job 18594604 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,722) - Starting with JobEnvironment(job_id=18594604, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594604/18594604_submitted.pkl
slurmstepd: error: *** JOB 18594604 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594665 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:26:47,443) - Starting with JobEnvironment(job_id=18594665, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:26:47,450) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594665/18594665_submitted.pkl
slurmstepd: error: *** JOB 18594665 ON n1719 CANCELLED AT 2025-07-23T16:27:15 DUE TO TIME LIMIT ***

⚠ Job 18594503 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:22:11,550) - Starting with JobEnvironment(job_id=18594503, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:22:11,552) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594503/18594503_submitted.pkl
slurmstepd: error: *** JOB 18594503 ON n1719 CANCELLED AT 2025-07-23T16:22:16 DUE TO TIME LIMIT ***

⚠ Job 18594600 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:25:15,721) - Starting with JobEnvironment(job_id=18594600, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:25:15,723) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594600/18594600_submitted.pkl
slurmstepd: error: *** JOB 18594600 ON n1719 CANCELLED AT 2025-07-23T16:25:15 DUE TO TIME LIMIT ***


⚠ Job 18594488 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:21:41,259) - Starting with JobEnvironment(job_id=18594488, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:21:41,260) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594488/18594488_submitted.pkl
slurmstepd: error: *** JOB 18594488 ON n1719 CANCELLED AT 2025-07-23T16:21:45 DUE TO TIME LIMIT ***


⚠ Job 18594551 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:23:13,431) - Starting with JobEnvironment(job_id=18594551, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:23:13,433) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594551/18594551_submitted.pkl
slurmstepd: error: *** JOB 18594551 ON n1719 CANCELLED AT 2025-07-23T16:23:15 DUE TO TIME LIMIT ***


⚠ Job 18594586 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_0_log.out
----------------------------------------
submitit INFO (2025-07-23 15:24:45,268) - Starting with JobEnvironment(job_id=18594586, hostname=n1719, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 15:24:45,271) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18594586/18594586_submitted.pkl
slurmstepd: error: *** JOB 18594586 ON n1719 CANCELLED AT 2025-07-23T16:24:45 DUE TO TIME LIMIT ***



BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            :  10%|▒░░░░░░░░░| 48/500 [4:32:54<34:22:26, 273.78s/it]
⚠ Job 18597175 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,389) - Starting with JobEnvironment(job_id=18597175, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,391) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597175/18597175_submitted.pkl
slurmstepd: error: *** JOB 18597175 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597116 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,590) - Starting with JobEnvironment(job_id=18597116, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597116/18597116_submitted.pkl
slurmstepd: error: *** JOB 18597116 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***


⚠ Job 18597174 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,388) - Starting with JobEnvironment(job_id=18597174, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,389) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597174/18597174_submitted.pkl
slurmstepd: error: *** JOB 18597174 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597159 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:06,971) - Starting with JobEnvironment(job_id=18597159, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:06,973) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597159/18597159_submitted.pkl
slurmstepd: error: *** JOB 18597159 ON n1720 CANCELLED AT 2025-07-23T18:05:32 DUE TO TIME LIMIT ***


⚠ Job 18597170 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:38,815) - Starting with JobEnvironment(job_id=18597170, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:38,817) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597170/18597170_submitted.pkl
slurmstepd: error: *** JOB 18597170 ON n1720 CANCELLED AT 2025-07-23T18:06:03 DUE TO TIME LIMIT ***


⚠ Job 18597122 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,589) - Starting with JobEnvironment(job_id=18597122, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597122/18597122_submitted.pkl
slurmstepd: error: *** JOB 18597122 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***


⚠ Job 18597128 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:29,698) - Starting with JobEnvironment(job_id=18597128, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:29,699) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597128/18597128_submitted.pkl
slurmstepd: error: *** JOB 18597128 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***


⚠ Job 18597129 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:28,896) - Starting with JobEnvironment(job_id=18597129, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:28,897) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597129/18597129_submitted.pkl
slurmstepd: error: *** JOB 18597129 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***


⚠ Job 18597126 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:03:28,924) - Starting with JobEnvironment(job_id=18597126, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:03:28,925) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597126/18597126_submitted.pkl
slurmstepd: error: *** JOB 18597126 ON n1720 CANCELLED AT 2025-07-23T18:03:32 DUE TO TIME LIMIT ***


⚠ Job 18597120 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:02:56,589) - Starting with JobEnvironment(job_id=18597120, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:02:56,591) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597120/18597120_submitted.pkl
slurmstepd: error: *** JOB 18597120 ON n1720 CANCELLED AT 2025-07-23T18:03:02 DUE TO TIME LIMIT ***

⚠ Job 18597176 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,389) - Starting with JobEnvironment(job_id=18597176, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,391) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597176/18597176_submitted.pkl
slurmstepd: error: *** JOB 18597176 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597235 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:09:27,570) - Starting with JobEnvironment(job_id=18597235, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:09:27,571) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597235/18597235_submitted.pkl
slurmstepd: error: *** JOB 18597235 ON n1720 CANCELLED AT 2025-07-23T18:09:32 DUE TO TIME LIMIT ***


⚠ Job 18597229 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,611) - Starting with JobEnvironment(job_id=18597229, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,612) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597229/18597229_submitted.pkl
slurmstepd: error: *** JOB 18597229 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***


⚠ Job 18597224 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:22,909) - Starting with JobEnvironment(job_id=18597224, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:22,912) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597224/18597224_submitted.pkl
slurmstepd: error: *** JOB 18597224 ON n1720 CANCELLED AT 2025-07-23T18:08:33 DUE TO TIME LIMIT ***


⚠ Job 18597167 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:05:38,815) - Starting with JobEnvironment(job_id=18597167, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:05:38,817) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597167/18597167_submitted.pkl
slurmstepd: error: *** JOB 18597167 ON n1720 CANCELLED AT 2025-07-23T18:06:03 DUE TO TIME LIMIT ***


⚠ Job 18597232 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,605) - Starting with JobEnvironment(job_id=18597232, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597232/18597232_submitted.pkl
slurmstepd: error: *** JOB 18597232 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***



⚠ Job 18597157 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:04:33,940) - Starting with JobEnvironment(job_id=18597157, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:04:33,943) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597157/18597157_submitted.pkl
slurmstepd: error: *** JOB 18597157 ON n1720 CANCELLED AT 2025-07-23T18:04:32 DUE TO TIME LIMIT ***


⚠ Job 18597216 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:17,206) - Starting with JobEnvironment(job_id=18597216, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:17,209) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597216/18597216_submitted.pkl
slurmstepd: error: *** JOB 18597216 ON n1720 CANCELLED AT 2025-07-23T18:07:32 DUE TO TIME LIMIT ***


⚠ Job 18597228 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,615) - Starting with JobEnvironment(job_id=18597228, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597228/18597228_submitted.pkl
slurmstepd: error: *** JOB 18597228 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***


⚠ Job 18597222 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:49,817) - Starting with JobEnvironment(job_id=18597222, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:49,819) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597222/18597222_submitted.pkl
slurmstepd: error: *** JOB 18597222 ON n1720 CANCELLED AT 2025-07-23T18:08:02 DUE TO TIME LIMIT ***


⚠ Job 18597213 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:44,831) - Starting with JobEnvironment(job_id=18597213, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:44,833) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597213/18597213_submitted.pkl
slurmstepd: error: *** JOB 18597213 ON n1720 CANCELLED AT 2025-07-23T18:07:02 DUE TO TIME LIMIT ***


⚠ Job 18597231 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,610) - Starting with JobEnvironment(job_id=18597231, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597231/18597231_submitted.pkl
slurmstepd: error: *** JOB 18597231 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***


⚠ Job 18597227 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,609) - Starting with JobEnvironment(job_id=18597227, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597227/18597227_submitted.pkl
slurmstepd: error: *** JOB 18597227 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***


⚠ Job 18597230 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:55,601) - Starting with JobEnvironment(job_id=18597230, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:55,604) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597230/18597230_submitted.pkl
slurmstepd: error: *** JOB 18597230 ON n1720 CANCELLED AT 2025-07-23T18:09:02 DUE TO TIME LIMIT ***


⚠ Job 18597177 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,396) - Starting with JobEnvironment(job_id=18597177, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,398) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597177/18597177_submitted.pkl
slurmstepd: error: *** JOB 18597177 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597217 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:07:18,103) - Starting with JobEnvironment(job_id=18597217, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:07:18,105) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597217/18597217_submitted.pkl
slurmstepd: error: *** JOB 18597217 ON n1720 CANCELLED AT 2025-07-23T18:07:32 DUE TO TIME LIMIT ***


⚠ Job 18597225 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:08:22,909) - Starting with JobEnvironment(job_id=18597225, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:08:22,912) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597225/18597225_submitted.pkl
slurmstepd: error: *** JOB 18597225 ON n1720 CANCELLED AT 2025-07-23T18:08:33 DUE TO TIME LIMIT ***


⚠ Job 18597171 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,421) - Starting with JobEnvironment(job_id=18597171, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597171/18597171_submitted.pkl
slurmstepd: error: *** JOB 18597171 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597172 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:11,388) - Starting with JobEnvironment(job_id=18597172, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:11,390) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597172/18597172_submitted.pkl
slurmstepd: error: *** JOB 18597172 ON n1720 CANCELLED AT 2025-07-23T18:06:32 DUE TO TIME LIMIT ***


⚠ Job 18597212 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_0_log.out
----------------------------------------
submitit INFO (2025-07-23 17:06:44,832) - Starting with JobEnvironment(job_id=18597212, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 17:06:44,836) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18597212/18597212_submitted.pkl
slurmstepd: error: *** JOB 18597212 ON n1720 CANCELLED AT 2025-07-23T18:07:02 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             :  16%|█▒░░░░░░░░| 78/500 [6:26:32<10:14:55, 87.43s/it]
⚠ Job 18598619 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,946) - Starting with JobEnvironment(job_id=18598619, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598619/18598619_submitted.pkl
slurmstepd: error: *** JOB 18598619 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***


⚠ Job 18598611 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,442) - Starting with JobEnvironment(job_id=18598611, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,444) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598611/18598611_submitted.pkl
slurmstepd: error: *** JOB 18598611 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***


⚠ Job 18598605 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,443) - Starting with JobEnvironment(job_id=18598605, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598605/18598605_submitted.pkl
slurmstepd: error: *** JOB 18598605 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***

⚠ Job 18598616 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,944) - Starting with JobEnvironment(job_id=18598616, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,945) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598616/18598616_submitted.pkl
slurmstepd: error: *** JOB 18598616 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***


⚠ Job 18598617 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:03:28,946) - Starting with JobEnvironment(job_id=18598617, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:03:28,947) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598617/18598617_submitted.pkl
slurmstepd: error: *** JOB 18598617 ON n1720 CANCELLED AT 2025-07-23T20:03:50 DUE TO TIME LIMIT ***


⚠ Job 18598613 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,442) - Starting with JobEnvironment(job_id=18598613, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598613/18598613_submitted.pkl
slurmstepd: error: *** JOB 18598613 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***



⚠ Job 18598612 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:02:57,443) - Starting with JobEnvironment(job_id=18598612, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:02:57,445) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598612/18598612_submitted.pkl
slurmstepd: error: *** JOB 18598612 ON n1720 CANCELLED AT 2025-07-23T20:03:20 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             :  16%|█▒░░░░░░░░| 78/500 [6:36:21<10:14:55, 87.43s/it]
⚠ Job 18598739 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:07:40,642) - Starting with JobEnvironment(job_id=18598739, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:07:40,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598739/18598739_submitted.pkl
slurmstepd: error: *** JOB 18598739 ON n1720 CANCELLED AT 2025-07-23T20:07:50 DUE TO TIME LIMIT ***


⚠ Job 18598748 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,435) - Starting with JobEnvironment(job_id=18598748, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598748/18598748_submitted.pkl
slurmstepd: error: *** JOB 18598748 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***


⚠ Job 18598811 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,402) - Starting with JobEnvironment(job_id=18598811, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,404) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598811/18598811_submitted.pkl
slurmstepd: error: *** JOB 18598811 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***


⚠ Job 18598746 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,436) - Starting with JobEnvironment(job_id=18598746, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598746/18598746_submitted.pkl
slurmstepd: error: *** JOB 18598746 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***


⚠ Job 18598848 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:58,660) - Starting with JobEnvironment(job_id=18598848, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:58,661) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598848/18598848_submitted.pkl
slurmstepd: error: *** JOB 18598848 ON n1720 CANCELLED AT 2025-07-23T20:14:21 DUE TO TIME LIMIT ***


⚠ Job 18598819 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,336) - Starting with JobEnvironment(job_id=18598819, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,338) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598819/18598819_submitted.pkl
slurmstepd: error: *** JOB 18598819 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***

⚠ Job 18598745 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,436) - Starting with JobEnvironment(job_id=18598745, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,437) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598745/18598745_submitted.pkl
slurmstepd: error: *** JOB 18598745 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***


⚠ Job 18598813 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:12:55,293) - Starting with JobEnvironment(job_id=18598813, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:12:55,295) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598813/18598813_submitted.pkl
slurmstepd: error: *** JOB 18598813 ON n1720 CANCELLED AT 2025-07-23T20:13:21 DUE TO TIME LIMIT ***


⚠ Job 18598820 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:26,841) - Starting with JobEnvironment(job_id=18598820, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:26,843) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598820/18598820_submitted.pkl
slurmstepd: error: *** JOB 18598820 ON n1720 CANCELLED AT 2025-07-23T20:13:51 DUE TO TIME LIMIT ***


⚠ Job 18598754 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:09:15,197) - Starting with JobEnvironment(job_id=18598754, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:09:15,200) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598754/18598754_submitted.pkl
slurmstepd: error: *** JOB 18598754 ON n1720 CANCELLED AT 2025-07-23T20:09:21 DUE TO TIME LIMIT ***



⚠ Job 18598821 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:13:26,742) - Starting with JobEnvironment(job_id=18598821, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:13:26,743) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598821/18598821_submitted.pkl
slurmstepd: error: *** JOB 18598821 ON n1720 CANCELLED AT 2025-07-23T20:13:51 DUE TO TIME LIMIT ***


⚠ Job 18598747 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:08:12,421) - Starting with JobEnvironment(job_id=18598747, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:08:12,423) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598747/18598747_submitted.pkl
slurmstepd: error: *** JOB 18598747 ON n1720 CANCELLED AT 2025-07-23T20:08:20 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 :  16%|█▒░░░░░░░░| 78/500 [6:47:23<10:14:55, 87.43s/it]
⚠ Job 18598851 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_0_log.out
----------------------------------------
submitit INFO (2025-07-23 19:17:06,853) - Starting with JobEnvironment(job_id=18598851, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 19:17:06,855) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18598851/18598851_submitted.pkl
slurmstepd: error: *** JOB 18598851 ON n1720 CANCELLED AT 2025-07-23T20:17:22 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             :  16%|█▒░░░░░░░░| 82/500 [8:54:58<104:50:12, 902.90s/it]
⚠ Job 18599279 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599279, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,522) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599279/18599279_submitted.pkl
slurmstepd: error: *** JOB 18599279 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***

⚠ Job 18599285 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,076) - Starting with JobEnvironment(job_id=18599285, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599285/18599285_submitted.pkl
slurmstepd: error: *** JOB 18599285 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***


⚠ Job 18599289 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,073) - Starting with JobEnvironment(job_id=18599289, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,075) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599289/18599289_submitted.pkl
slurmstepd: error: *** JOB 18599289 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***


⚠ Job 18599283 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,729) - Starting with JobEnvironment(job_id=18599283, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,732) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599283/18599283_submitted.pkl
slurmstepd: error: *** JOB 18599283 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


⚠ Job 18599287 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,073) - Starting with JobEnvironment(job_id=18599287, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,076) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599287/18599287_submitted.pkl
slurmstepd: error: *** JOB 18599287 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599296 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599296, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599296/18599296_submitted.pkl
slurmstepd: error: *** JOB 18599296 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599293 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:27:20,757) - Starting with JobEnvironment(job_id=18599293, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:27:20,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599293/18599293_submitted.pkl
slurmstepd: error: *** JOB 18599293 ON n1720 CANCELLED AT 2025-07-23T22:27:35 DUE TO TIME LIMIT ***


⚠ Job 18599310 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599310, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599310/18599310_submitted.pkl
slurmstepd: error: *** JOB 18599310 ON n1720 CANCELLED AT 2025-07-23T22:31:37 DUE TO TIME LIMIT ***


⚠ Job 18599280 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599280, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599280/18599280_submitted.pkl
slurmstepd: error: *** JOB 18599280 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


⚠ Job 18599295 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599295, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599295/18599295_submitted.pkl
slurmstepd: error: *** JOB 18599295 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599308 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599308, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599308/18599308_submitted.pkl
slurmstepd: error: *** JOB 18599308 ON n1720 CANCELLED AT 2025-07-23T22:31:36 DUE TO TIME LIMIT ***


⚠ Job 18599304 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599304, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599304/18599304_submitted.pkl
slurmstepd: error: *** JOB 18599304 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599309 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,802) - Starting with JobEnvironment(job_id=18599309, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599309/18599309_submitted.pkl
slurmstepd: error: *** JOB 18599309 ON n1720 CANCELLED AT 2025-07-23T22:31:37 DUE TO TIME LIMIT ***


⚠ Job 18599307 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:31:11,803) - Starting with JobEnvironment(job_id=18599307, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:31:11,805) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599307/18599307_submitted.pkl
slurmstepd: error: *** JOB 18599307 ON n1720 CANCELLED AT 2025-07-23T22:31:36 DUE TO TIME LIMIT ***


⚠ Job 18599284 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599284, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599284/18599284_submitted.pkl
slurmstepd: error: *** JOB 18599284 ON n1720 CANCELLED AT 2025-07-23T22:25:06 DUE TO TIME LIMIT ***

⚠ Job 18599298 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599298, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599298/18599298_submitted.pkl
slurmstepd: error: *** JOB 18599298 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599277 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599277, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599277/18599277_submitted.pkl
slurmstepd: error: *** JOB 18599277 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


⚠ Job 18599282 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599282, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599282/18599282_submitted.pkl
slurmstepd: error: *** JOB 18599282 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


⚠ Job 18599300 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599300, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599300/18599300_submitted.pkl
slurmstepd: error: *** JOB 18599300 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599305 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:30:40,574) - Starting with JobEnvironment(job_id=18599305, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:30:40,577) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599305/18599305_submitted.pkl
slurmstepd: error: *** JOB 18599305 ON n1720 CANCELLED AT 2025-07-23T22:30:35 DUE TO TIME LIMIT ***


⚠ Job 18599275 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599275, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599275/18599275_submitted.pkl
slurmstepd: error: *** JOB 18599275 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***

⚠ Job 18599302 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,967) - Starting with JobEnvironment(job_id=18599302, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,970) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599302/18599302_submitted.pkl
slurmstepd: error: *** JOB 18599302 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599290 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,075) - Starting with JobEnvironment(job_id=18599290, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,076) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599290/18599290_submitted.pkl
slurmstepd: error: *** JOB 18599290 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***


⚠ Job 18599301 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,951) - Starting with JobEnvironment(job_id=18599301, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,954) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599301/18599301_submitted.pkl
slurmstepd: error: *** JOB 18599301 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599286 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,075) - Starting with JobEnvironment(job_id=18599286, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,077) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599286/18599286_submitted.pkl
slurmstepd: error: *** JOB 18599286 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***


⚠ Job 18599306 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:30:40,574) - Starting with JobEnvironment(job_id=18599306, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:30:40,577) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599306/18599306_submitted.pkl
slurmstepd: error: *** JOB 18599306 ON n1720 CANCELLED AT 2025-07-23T22:30:35 DUE TO TIME LIMIT ***


⚠ Job 18599278 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,519) - Starting with JobEnvironment(job_id=18599278, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,522) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599278/18599278_submitted.pkl
slurmstepd: error: *** JOB 18599278 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


⚠ Job 18599292 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:26:49,167) - Starting with JobEnvironment(job_id=18599292, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:26:49,170) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599292/18599292_submitted.pkl
slurmstepd: error: *** JOB 18599292 ON n1720 CANCELLED AT 2025-07-23T22:27:05 DUE TO TIME LIMIT ***


⚠ Job 18599294 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:27:20,757) - Starting with JobEnvironment(job_id=18599294, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:27:20,760) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599294/18599294_submitted.pkl
slurmstepd: error: *** JOB 18599294 ON n1720 CANCELLED AT 2025-07-23T22:27:35 DUE TO TIME LIMIT ***


⚠ Job 18599297 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:28:59,952) - Starting with JobEnvironment(job_id=18599297, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:28:59,953) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599297/18599297_submitted.pkl
slurmstepd: error: *** JOB 18599297 ON n1720 CANCELLED AT 2025-07-23T22:29:05 DUE TO TIME LIMIT ***


⚠ Job 18599288 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:25:09,072) - Starting with JobEnvironment(job_id=18599288, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:25:09,075) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599288/18599288_submitted.pkl
slurmstepd: error: *** JOB 18599288 ON n1720 CANCELLED AT 2025-07-23T22:25:35 DUE TO TIME LIMIT ***



⚠ Job 18599281 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:24:37,520) - Starting with JobEnvironment(job_id=18599281, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:24:37,521) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599281/18599281_submitted.pkl
slurmstepd: error: *** JOB 18599281 ON n1720 CANCELLED AT 2025-07-23T22:25:05 DUE TO TIME LIMIT ***


BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            :  16%|█▒░░░░░░░░| 82/500 [9:02:32<104:50:12, 902.90s/it]
⚠ Job 18599314 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,752) - Starting with JobEnvironment(job_id=18599314, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,754) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599314/18599314_submitted.pkl
slurmstepd: error: *** JOB 18599314 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            :  16%|█▒░░░░░░░░| 82/500 [9:06:46<104:50:12, 902.90s/it]
⚠ Job 18599315 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599315, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599315/18599315_submitted.pkl
slurmstepd: error: *** JOB 18599315 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599316 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599316, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599316/18599316_submitted.pkl
slurmstepd: error: *** JOB 18599316 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599317 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599317, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599317/18599317_submitted.pkl
slurmstepd: error: *** JOB 18599317 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599319 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599319, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599319/18599319_submitted.pkl
slurmstepd: error: *** JOB 18599319 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599321 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599321, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599321/18599321_submitted.pkl
slurmstepd: error: *** JOB 18599321 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599320 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:33:21,745) - Starting with JobEnvironment(job_id=18599320, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:33:21,747) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599320/18599320_submitted.pkl
slurmstepd: error: *** JOB 18599320 ON n1720 CANCELLED AT 2025-07-23T22:33:35 DUE TO TIME LIMIT ***


⚠ Job 18599322 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:26,435) - Starting with JobEnvironment(job_id=18599322, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:26,438) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599322/18599322_submitted.pkl
slurmstepd: error: *** JOB 18599322 ON n1720 CANCELLED AT 2025-07-23T22:34:36 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 154, best VAL_ACC: 98.44, timeout/running 9/5 = ∑14/50, job_failed                             :  16%|█▒░░░░░░░░| 82/500 [9:07:11<104:50:12, 902.90s/it]
⚠ Job 18599323 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:31,599) - Starting with JobEnvironment(job_id=18599323, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:31,603) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599323/18599323_submitted.pkl
slurmstepd: error: *** JOB 18599323 ON n1720 CANCELLED AT 2025-07-23T22:34:36 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                :  16%|█▒░░░░░░░░| 82/500 [9:08:48<104:50:12, 902.90s/it]
⚠ Job 18599326 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,614) - Starting with JobEnvironment(job_id=18599326, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599326/18599326_submitted.pkl
slurmstepd: error: *** JOB 18599326 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***


⚠ Job 18599324 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:59,285) - Starting with JobEnvironment(job_id=18599324, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:59,288) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599324/18599324_submitted.pkl
slurmstepd: error: *** JOB 18599324 ON n1720 CANCELLED AT 2025-07-23T22:35:05 DUE TO TIME LIMIT ***


⚠ Job 18599328 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,610) - Starting with JobEnvironment(job_id=18599328, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,613) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599328/18599328_submitted.pkl
slurmstepd: error: *** JOB 18599328 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***


⚠ Job 18599325 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:34:58,238) - Starting with JobEnvironment(job_id=18599325, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:34:58,241) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599325/18599325_submitted.pkl
slurmstepd: error: *** JOB 18599325 ON n1720 CANCELLED AT 2025-07-23T22:35:05 DUE TO TIME LIMIT ***


⚠ Job 18599327 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_0_log.out
----------------------------------------
submitit INFO (2025-07-23 21:35:30,614) - Starting with JobEnvironment(job_id=18599327, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 21:35:30,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599327/18599327_submitted.pkl
slurmstepd: error: *** JOB 18599327 ON n1720 CANCELLED AT 2025-07-23T22:35:35 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 169 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             :  18%|█▒░░░░░░░░| 88/500 [11:17:20<104:14:08, 910.80s/it]
⚠ Job 18599511 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599511, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599511/18599511_submitted.pkl
slurmstepd: error: *** JOB 18599511 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599513 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599513, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599513/18599513_submitted.pkl
slurmstepd: error: *** JOB 18599513 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599514 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599514, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599514/18599514_submitted.pkl
slurmstepd: error: *** JOB 18599514 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599518 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599518, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599518/18599518_submitted.pkl
slurmstepd: error: *** JOB 18599518 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599515 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599515, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599515/18599515_submitted.pkl
slurmstepd: error: *** JOB 18599515 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599524 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599524, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599524/18599524_submitted.pkl
slurmstepd: error: *** JOB 18599524 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***


⚠ Job 18599516 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599516, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599516/18599516_submitted.pkl
slurmstepd: error: *** JOB 18599516 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599510 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599510, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599510/18599510_submitted.pkl
slurmstepd: error: *** JOB 18599510 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***


⚠ Job 18599512 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599512, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599512/18599512_submitted.pkl
slurmstepd: error: *** JOB 18599512 ON n1720 CANCELLED AT 2025-07-24T00:51:19 DUE TO TIME LIMIT ***


⚠ Job 18599520 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,523) - Starting with JobEnvironment(job_id=18599520, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,525) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599520/18599520_submitted.pkl
slurmstepd: error: *** JOB 18599520 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***

⚠ Job 18599509 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,643) - Starting with JobEnvironment(job_id=18599509, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,645) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599509/18599509_submitted.pkl
slurmstepd: error: *** JOB 18599509 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***

⚠ Job 18599522 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599522, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599522/18599522_submitted.pkl
slurmstepd: error: *** JOB 18599522 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***




⚠ Job 18599523 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:51:29,524) - Starting with JobEnvironment(job_id=18599523, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:51:29,526) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599523/18599523_submitted.pkl
slurmstepd: error: *** JOB 18599523 ON n1720 CANCELLED AT 2025-07-24T00:51:48 DUE TO TIME LIMIT ***


⚠ Job 18599517 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_0_log.out
----------------------------------------
submitit INFO (2025-07-23 23:50:58,642) - Starting with JobEnvironment(job_id=18599517, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-23 23:50:58,644) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599517/18599517_submitted.pkl
slurmstepd: error: *** JOB 18599517 ON n1720 CANCELLED AT 2025-07-24T00:51:18 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 183 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             :  18%|█▒░░░░░░░░| 88/500 [11:32:22<104:14:08, 910.80s/it]
⚠ Job 18599551 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,679) - Starting with JobEnvironment(job_id=18599551, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,683) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599551/18599551_submitted.pkl
slurmstepd: error: *** JOB 18599551 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599561 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:08:57,938) - Starting with JobEnvironment(job_id=18599561, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:08:57,940) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599561/18599561_submitted.pkl
slurmstepd: error: *** JOB 18599561 ON n1720 CANCELLED AT 2025-07-24T01:09:18 DUE TO TIME LIMIT ***


⚠ Job 18599555 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,560) - Starting with JobEnvironment(job_id=18599555, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,563) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599555/18599555_submitted.pkl
slurmstepd: error: *** JOB 18599555 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599554 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,646) - Starting with JobEnvironment(job_id=18599554, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599554/18599554_submitted.pkl
slurmstepd: error: *** JOB 18599554 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599559 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,521) - Starting with JobEnvironment(job_id=18599559, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,523) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599559/18599559_submitted.pkl
slurmstepd: error: *** JOB 18599559 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599552 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,687) - Starting with JobEnvironment(job_id=18599552, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,689) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599552/18599552_submitted.pkl
slurmstepd: error: *** JOB 18599552 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***

⚠ Job 18599549 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:10,772) - Starting with JobEnvironment(job_id=18599549, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:10,775) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599549/18599549_submitted.pkl
slurmstepd: error: *** JOB 18599549 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***


⚠ Job 18599547 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:10,772) - Starting with JobEnvironment(job_id=18599547, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:10,774) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599547/18599547_submitted.pkl
slurmstepd: error: *** JOB 18599547 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***


⚠ Job 18599553 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,645) - Starting with JobEnvironment(job_id=18599553, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,649) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599553/18599553_submitted.pkl
slurmstepd: error: *** JOB 18599553 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599556 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,471) - Starting with JobEnvironment(job_id=18599556, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,473) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599556/18599556_submitted.pkl
slurmstepd: error: *** JOB 18599556 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599560 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,467) - Starting with JobEnvironment(job_id=18599560, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,472) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599560/18599560_submitted.pkl
slurmstepd: error: *** JOB 18599560 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***


⚠ Job 18599550 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:15,761) - Starting with JobEnvironment(job_id=18599550, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:15,774) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599550/18599550_submitted.pkl
slurmstepd: error: *** JOB 18599550 ON n1720 CANCELLED AT 2025-07-24T01:05:18 DUE TO TIME LIMIT ***

⚠ Job 18599558 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:05:42,520) - Starting with JobEnvironment(job_id=18599558, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:05:42,523) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599558/18599558_submitted.pkl
slurmstepd: error: *** JOB 18599558 ON n1720 CANCELLED AT 2025-07-24T01:05:49 DUE TO TIME LIMIT ***




⚠ Job 18599562 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_0_log.out
----------------------------------------
submitit INFO (2025-07-24 00:08:57,938) - Starting with JobEnvironment(job_id=18599562, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 00:08:57,952) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599562/18599562_submitted.pkl
slurmstepd: error: *** JOB 18599562 ON n1720 CANCELLED AT 2025-07-24T01:09:18 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 197 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             :  18%|█▒░░░░░░░░| 91/500 [13:45:50<199:46:51, 1758.46s/it]
⚠ Job 18599794 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599794, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,343) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599794/18599794_submitted.pkl
slurmstepd: error: *** JOB 18599794 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599787 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599787, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599787/18599787_submitted.pkl
slurmstepd: error: *** JOB 18599787 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599792 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599792, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599792/18599792_submitted.pkl
slurmstepd: error: *** JOB 18599792 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599789 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599789, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599789/18599789_submitted.pkl
slurmstepd: error: *** JOB 18599789 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599781 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599781, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599781/18599781_submitted.pkl
slurmstepd: error: *** JOB 18599781 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599791 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599791, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599791/18599791_submitted.pkl
slurmstepd: error: *** JOB 18599791 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599775 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599775, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599775/18599775_submitted.pkl
slurmstepd: error: *** JOB 18599775 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599797 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599797, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599797/18599797_submitted.pkl
slurmstepd: error: *** JOB 18599797 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599777 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599777, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599777/18599777_submitted.pkl
slurmstepd: error: *** JOB 18599777 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599782 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599782, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,557) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599782/18599782_submitted.pkl
slurmstepd: error: *** JOB 18599782 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599779 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599779, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599779/18599779_submitted.pkl
slurmstepd: error: *** JOB 18599779 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599786 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599786, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599786/18599786_submitted.pkl
slurmstepd: error: *** JOB 18599786 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599790 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599790, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,616) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599790/18599790_submitted.pkl
slurmstepd: error: *** JOB 18599790 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599798 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599798, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,343) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599798/18599798_submitted.pkl
slurmstepd: error: *** JOB 18599798 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599784 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599784, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599784/18599784_submitted.pkl
slurmstepd: error: *** JOB 18599784 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599806 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599806, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599806/18599806_submitted.pkl
slurmstepd: error: *** JOB 18599806 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***


⚠ Job 18599783 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599783, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599783/18599783_submitted.pkl
slurmstepd: error: *** JOB 18599783 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599805 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,735) - Starting with JobEnvironment(job_id=18599805, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599805/18599805_submitted.pkl
slurmstepd: error: *** JOB 18599805 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***


⚠ Job 18599776 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599776, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599776/18599776_submitted.pkl
slurmstepd: error: *** JOB 18599776 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599800 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599800, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599800/18599800_submitted.pkl
slurmstepd: error: *** JOB 18599800 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***

⚠ Job 18599801 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,735) - Starting with JobEnvironment(job_id=18599801, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,748) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599801/18599801_submitted.pkl
slurmstepd: error: *** JOB 18599801 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***


⚠ Job 18599804 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599804, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599804/18599804_submitted.pkl
slurmstepd: error: *** JOB 18599804 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***


⚠ Job 18599788 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,603) - Starting with JobEnvironment(job_id=18599788, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,605) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599788/18599788_submitted.pkl
slurmstepd: error: *** JOB 18599788 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599803 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599803, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599803/18599803_submitted.pkl
slurmstepd: error: *** JOB 18599803 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***


⚠ Job 18599799 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599799, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599799/18599799_submitted.pkl
slurmstepd: error: *** JOB 18599799 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599802 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:54,734) - Starting with JobEnvironment(job_id=18599802, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:54,736) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599802/18599802_submitted.pkl
slurmstepd: error: *** JOB 18599802 ON n1720 CANCELLED AT 2025-07-24T03:27:55 DUE TO TIME LIMIT ***



⚠ Job 18599785 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:42,604) - Starting with JobEnvironment(job_id=18599785, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:42,606) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599785/18599785_submitted.pkl
slurmstepd: error: *** JOB 18599785 ON n1720 CANCELLED AT 2025-07-24T03:22:55 DUE TO TIME LIMIT ***


⚠ Job 18599793 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599793, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,342) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599793/18599793_submitted.pkl
slurmstepd: error: *** JOB 18599793 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599778 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,555) - Starting with JobEnvironment(job_id=18599778, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599778/18599778_submitted.pkl
slurmstepd: error: *** JOB 18599778 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***


⚠ Job 18599795 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599795, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599795/18599795_submitted.pkl
slurmstepd: error: *** JOB 18599795 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599796 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:27:24,340) - Starting with JobEnvironment(job_id=18599796, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:27:24,349) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599796/18599796_submitted.pkl
slurmstepd: error: *** JOB 18599796 ON n1720 CANCELLED AT 2025-07-24T03:27:25 DUE TO TIME LIMIT ***


⚠ Job 18599780 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:22:13,554) - Starting with JobEnvironment(job_id=18599780, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:22:13,556) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599780/18599780_submitted.pkl
slurmstepd: error: *** JOB 18599780 ON n1720 CANCELLED AT 2025-07-24T03:22:25 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 197 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           :  18%|█▒░░░░░░░░| 91/500 [13:59:05<199:46:51, 1758.46s/it]
⚠ Job 18599807 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599807, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599807/18599807_submitted.pkl
slurmstepd: error: *** JOB 18599807 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***


⚠ Job 18599808 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599808, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,632) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599808/18599808_submitted.pkl
slurmstepd: error: *** JOB 18599808 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***


⚠ Job 18599809 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:30:30,623) - Starting with JobEnvironment(job_id=18599809, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:30:30,626) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599809/18599809_submitted.pkl
slurmstepd: error: *** JOB 18599809 ON n1720 CANCELLED AT 2025-07-24T03:30:55 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 232 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             :  18%|█▒░░░░░░░░| 91/500 [14:06:37<199:46:51, 1758.46s/it]
⚠ Job 18599810 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599810, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599810/18599810_submitted.pkl
slurmstepd: error: *** JOB 18599810 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***


⚠ Job 18599814 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599814, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,966) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599814/18599814_submitted.pkl
slurmstepd: error: *** JOB 18599814 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***


⚠ Job 18599818 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599818, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,970) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599818/18599818_submitted.pkl
slurmstepd: error: *** JOB 18599818 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***


⚠ Job 18599815 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,956) - Starting with JobEnvironment(job_id=18599815, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,963) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599815/18599815_submitted.pkl
slurmstepd: error: *** JOB 18599815 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***


⚠ Job 18599812 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599812, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599812/18599812_submitted.pkl
slurmstepd: error: *** JOB 18599812 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***


⚠ Job 18599811 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:32:35,646) - Starting with JobEnvironment(job_id=18599811, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:32:35,648) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599811/18599811_submitted.pkl
slurmstepd: error: *** JOB 18599811 ON n1720 CANCELLED AT 2025-07-24T03:32:55 DUE TO TIME LIMIT ***


⚠ Job 18599816 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,956) - Starting with JobEnvironment(job_id=18599816, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599816/18599816_submitted.pkl
slurmstepd: error: *** JOB 18599816 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***

⚠ Job 18599822 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599822, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599822/18599822_submitted.pkl
slurmstepd: error: *** JOB 18599822 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



⚠ Job 18599817 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,953) - Starting with JobEnvironment(job_id=18599817, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,957) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599817/18599817_submitted.pkl
slurmstepd: error: *** JOB 18599817 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***

⚠ Job 18599823 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:35:42,717) - Starting with JobEnvironment(job_id=18599823, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:35:42,721) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599823/18599823_submitted.pkl
slurmstepd: error: *** JOB 18599823 ON n1720 CANCELLED AT 2025-07-24T03:35:56 DUE TO TIME LIMIT ***


⚠ Job 18599824 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:35:42,717) - Starting with JobEnvironment(job_id=18599824, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:35:42,727) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599824/18599824_submitted.pkl
slurmstepd: error: *** JOB 18599824 ON n1720 CANCELLED AT 2025-07-24T03:35:56 DUE TO TIME LIMIT ***

⚠ Job 18599819 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_0_log.out
----------------------------------------
submitit INFO (2025-07-24 02:33:05,955) - Starting with JobEnvironment(job_id=18599819, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 02:33:05,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18599819/18599819_submitted.pkl
slurmstepd: error: *** JOB 18599819 ON n1720 CANCELLED AT 2025-07-24T03:33:25 DUE TO TIME LIMIT ***



BoTorchGenerator, failed: 245 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             :  19%|█▒░░░░░░░░| 95/500 [16:43:24<202:47:34, 1802.60s/it]
⚠ Job 18600074 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,601) - Starting with JobEnvironment(job_id=18600074, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,603) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600074/18600074_submitted.pkl
slurmstepd: error: *** JOB 18600074 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600072 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600072, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600072/18600072_submitted.pkl
slurmstepd: error: *** JOB 18600072 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***


⚠ Job 18600068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600068, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600068/18600068_submitted.pkl
slurmstepd: error: *** JOB 18600068 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***


⚠ Job 18600070 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,982) - Starting with JobEnvironment(job_id=18600070, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,984) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600070/18600070_submitted.pkl
slurmstepd: error: *** JOB 18600070 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***


⚠ Job 18600079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:15,183) - Starting with JobEnvironment(job_id=18600079, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:15,190) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600079/18600079_submitted.pkl
slurmstepd: error: *** JOB 18600079 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600080 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,320) - Starting with JobEnvironment(job_id=18600080, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,323) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600080/18600080_submitted.pkl
slurmstepd: error: *** JOB 18600080 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***

⚠ Job 18600081 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,586) - Starting with JobEnvironment(job_id=18600081, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,588) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600081/18600081_submitted.pkl
slurmstepd: error: *** JOB 18600081 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,303) - Starting with JobEnvironment(job_id=18600075, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,305) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600075/18600075_submitted.pkl
slurmstepd: error: *** JOB 18600075 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***



⚠ Job 18600073 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,598) - Starting with JobEnvironment(job_id=18600073, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600073/18600073_submitted.pkl
slurmstepd: error: *** JOB 18600073 ON n1720 CANCELLED AT 2025-07-24T06:21:14 DUE TO TIME LIMIT ***


⚠ Job 18600076 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,302) - Starting with JobEnvironment(job_id=18600076, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,305) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600076/18600076_submitted.pkl
slurmstepd: error: *** JOB 18600076 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600078 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,338) - Starting with JobEnvironment(job_id=18600078, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,340) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600078/18600078_submitted.pkl
slurmstepd: error: *** JOB 18600078 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600077 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,574) - Starting with JobEnvironment(job_id=18600077, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,578) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600077/18600077_submitted.pkl
slurmstepd: error: *** JOB 18600077 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600067 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:20:42,976) - Starting with JobEnvironment(job_id=18600067, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:20:42,979) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600067/18600067_submitted.pkl
slurmstepd: error: *** JOB 18600067 ON n1720 CANCELLED AT 2025-07-24T06:20:43 DUE TO TIME LIMIT ***


⚠ Job 18600082 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:21:14,324) - Starting with JobEnvironment(job_id=18600082, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:21:14,325) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600082/18600082_submitted.pkl
slurmstepd: error: *** JOB 18600082 ON n1720 CANCELLED AT 2025-07-24T06:21:13 DUE TO TIME LIMIT ***


⚠ Job 18600066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:15:57,782) - Starting with JobEnvironment(job_id=18600066, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:15:57,791) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600066/18600066_submitted.pkl
slurmstepd: error: *** JOB 18600066 ON n1720 CANCELLED AT 2025-07-24T06:16:09 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 260 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                :  19%|█▒░░░░░░░░| 95/500 [16:58:53<202:47:34, 1802.60s/it]
⚠ Job 18600083 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:25:59,608) - Starting with JobEnvironment(job_id=18600083, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:25:59,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600083/18600083_submitted.pkl
slurmstepd: error: *** JOB 18600083 ON n1720 CANCELLED AT 2025-07-24T06:26:14 DUE TO TIME LIMIT ***


⚠ Job 18600084 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_0_log.out
----------------------------------------
submitit INFO (2025-07-24 05:25:59,608) - Starting with JobEnvironment(job_id=18600084, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 05:25:59,611) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600084/18600084_submitted.pkl
slurmstepd: error: *** JOB 18600084 ON n1720 CANCELLED AT 2025-07-24T06:26:15 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 262 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             :  19%|█▒░░░░░░░░| 95/500 [18:29:57<202:47:34, 1802.60s/it]
⚠ Job 18600238 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600238, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600238/18600238_submitted.pkl
slurmstepd: error: *** JOB 18600238 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600239 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600239, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600239/18600239_submitted.pkl
slurmstepd: error: *** JOB 18600239 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600243 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600243, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600243/18600243_submitted.pkl
slurmstepd: error: *** JOB 18600243 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600240 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600240, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600240/18600240_submitted.pkl
slurmstepd: error: *** JOB 18600240 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600241 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600241, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600241/18600241_submitted.pkl
slurmstepd: error: *** JOB 18600241 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600252 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600252, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600252/18600252_submitted.pkl
slurmstepd: error: *** JOB 18600252 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***


⚠ Job 18600250 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600250, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600250/18600250_submitted.pkl
slurmstepd: error: *** JOB 18600250 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***

⚠ Job 18600247 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600247, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,476) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600247/18600247_submitted.pkl
slurmstepd: error: *** JOB 18600247 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600251 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,061) - Starting with JobEnvironment(job_id=18600251, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,064) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600251/18600251_submitted.pkl
slurmstepd: error: *** JOB 18600251 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***


⚠ Job 18600255 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:48:32,377) - Starting with JobEnvironment(job_id=18600255, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:48:32,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600255/18600255_submitted.pkl
slurmstepd: error: *** JOB 18600255 ON n1720 CANCELLED AT 2025-07-24T08:48:57 DUE TO TIME LIMIT ***

⚠ Job 18600248 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,062) - Starting with JobEnvironment(job_id=18600248, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,065) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600248/18600248_submitted.pkl
slurmstepd: error: *** JOB 18600248 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***


⚠ Job 18600262 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,789) - Starting with JobEnvironment(job_id=18600262, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600262/18600262_submitted.pkl
slurmstepd: error: *** JOB 18600262 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600267 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,696) - Starting with JobEnvironment(job_id=18600267, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,698) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600267/18600267_submitted.pkl
slurmstepd: error: *** JOB 18600267 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600263 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600263, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600263/18600263_submitted.pkl
slurmstepd: error: *** JOB 18600263 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600246 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600246, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600246/18600246_submitted.pkl
slurmstepd: error: *** JOB 18600246 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***

⚠ Job 18600249 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:45:23,063) - Starting with JobEnvironment(job_id=18600249, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:45:23,065) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600249/18600249_submitted.pkl
slurmstepd: error: *** JOB 18600249 ON n1720 CANCELLED AT 2025-07-24T08:45:27 DUE TO TIME LIMIT ***


⚠ Job 18600268 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,696) - Starting with JobEnvironment(job_id=18600268, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,698) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600268/18600268_submitted.pkl
slurmstepd: error: *** JOB 18600268 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600259 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,698) - Starting with JobEnvironment(job_id=18600259, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,699) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600259/18600259_submitted.pkl
slurmstepd: error: *** JOB 18600259 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***

⚠ Job 18600266 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,689) - Starting with JobEnvironment(job_id=18600266, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,697) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600266/18600266_submitted.pkl
slurmstepd: error: *** JOB 18600266 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600261 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600261, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600261/18600261_submitted.pkl
slurmstepd: error: *** JOB 18600261 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600244 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600244, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600244/18600244_submitted.pkl
slurmstepd: error: *** JOB 18600244 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600242 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,473) - Starting with JobEnvironment(job_id=18600242, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600242/18600242_submitted.pkl
slurmstepd: error: *** JOB 18600242 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***

⚠ Job 18600237 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:22,397) - Starting with JobEnvironment(job_id=18600237, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:22,400) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600237/18600237_submitted.pkl
slurmstepd: error: *** JOB 18600237 ON n1720 CANCELLED AT 2025-07-24T08:44:27 DUE TO TIME LIMIT ***

⚠ Job 18600245 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:44:51,474) - Starting with JobEnvironment(job_id=18600245, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:44:51,475) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600245/18600245_submitted.pkl
slurmstepd: error: *** JOB 18600245 ON n1720 CANCELLED AT 2025-07-24T08:44:57 DUE TO TIME LIMIT ***


⚠ Job 18600260 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600260, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600260/18600260_submitted.pkl
slurmstepd: error: *** JOB 18600260 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


⚠ Job 18600254 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:48:32,377) - Starting with JobEnvironment(job_id=18600254, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:48:32,380) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600254/18600254_submitted.pkl
slurmstepd: error: *** JOB 18600254 ON n1720 CANCELLED AT 2025-07-24T08:48:57 DUE TO TIME LIMIT ***





⚠ Job 18600265 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600265, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600265/18600265_submitted.pkl
slurmstepd: error: *** JOB 18600265 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***



⚠ Job 18600258 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:09,538) - Starting with JobEnvironment(job_id=18600258, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:09,541) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600258/18600258_submitted.pkl
slurmstepd: error: *** JOB 18600258 ON n1720 CANCELLED AT 2025-07-24T08:51:29 DUE TO TIME LIMIT ***


⚠ Job 18600264 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_0_log.out
----------------------------------------
submitit INFO (2025-07-24 07:51:40,709) - Starting with JobEnvironment(job_id=18600264, hostname=n1720, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 07:51:40,711) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600264/18600264_submitted.pkl
slurmstepd: error: *** JOB 18600264 ON n1720 CANCELLED AT 2025-07-24T08:51:58 DUE TO TIME LIMIT ***


BoTorchGenerator, failed: 291 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             :  19%|█▒░░░░░░░░| 97/500 [21:58:14<553:27:49, 4944.09s/it]
⚠ Job 18600976 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,598) - Starting with JobEnvironment(job_id=18600976, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,600) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600976/18600976_submitted.pkl
slurmstepd: error: *** JOB 18600976 ON n1178 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600969 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:26,744) - Starting with JobEnvironment(job_id=18600969, hostname=n1268, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:26,746) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600969/18600969_submitted.pkl
slurmstepd: error: *** JOB 18600969 ON n1268 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600972 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,157) - Starting with JobEnvironment(job_id=18600972, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600972/18600972_submitted.pkl
slurmstepd: error: *** JOB 18600972 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600970 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:26,743) - Starting with JobEnvironment(job_id=18600970, hostname=n1268, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:26,746) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600970/18600970_submitted.pkl
slurmstepd: error: *** JOB 18600970 ON n1268 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600975 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600975, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600975/18600975_submitted.pkl
slurmstepd: error: *** JOB 18600975 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600971 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600971, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,158) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600971/18600971_submitted.pkl
slurmstepd: error: *** JOB 18600971 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600967 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,216) - Starting with JobEnvironment(job_id=18600967, hostname=n1270, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,219) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600967/18600967_submitted.pkl
slurmstepd: error: *** JOB 18600967 ON n1270 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***

⚠ Job 18600968 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,216) - Starting with JobEnvironment(job_id=18600968, hostname=n1270, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,218) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600968/18600968_submitted.pkl
slurmstepd: error: *** JOB 18600968 ON n1270 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600973 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600973, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,159) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600973/18600973_submitted.pkl
slurmstepd: error: *** JOB 18600973 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


⚠ Job 18600974 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:37:27,156) - Starting with JobEnvironment(job_id=18600974, hostname=n1218, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:37:27,159) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600974/18600974_submitted.pkl
slurmstepd: error: *** JOB 18600974 ON n1218 CANCELLED AT 2025-07-24T11:37:47 DUE TO TIME LIMIT ***


BoTorchGenerator, failed: 301 ('VAL_ACC: ' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, new result: 98.47                         :  20%|█▒░░░░░░░░| 98/500 [22:16:00<422:14:16, 3781.24s/it]
⚠ Job 18600982 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600982, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600982/18600982_submitted.pkl
slurmstepd: error: *** JOB 18600982 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***


⚠ Job 18600980 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600980, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600980/18600980_submitted.pkl
slurmstepd: error: *** JOB 18600980 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***


⚠ Job 18600981 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600981, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600981/18600981_submitted.pkl
slurmstepd: error: *** JOB 18600981 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***


⚠ Job 18600978 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600978, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,796) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600978/18600978_submitted.pkl
slurmstepd: error: *** JOB 18600978 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***


⚠ Job 18600977 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600977, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,797) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600977/18600977_submitted.pkl
slurmstepd: error: *** JOB 18600977 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***


⚠ Job 18600979 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_0_log.out
----------------------------------------
submitit INFO (2025-07-24 10:38:30,795) - Starting with JobEnvironment(job_id=18600979, hostname=n1178, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 10:38:30,796) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18600979/18600979_submitted.pkl
slurmstepd: error: *** JOB 18600979 ON n1178 CANCELLED AT 2025-07-24T11:38:48 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 307 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             :  20%|██░░░░░░░░| 100/500 [22:19:37<299:40:38, 2697.10s/it]
⚠ Job 18601071 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601071, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601071/18601071_submitted.pkl
slurmstepd: error: *** JOB 18601071 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***


⚠ Job 18601052 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,993) - Starting with JobEnvironment(job_id=18601052, hostname=n1626, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,995) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601052/18601052_submitted.pkl
slurmstepd: error: *** JOB 18601052 ON n1626 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***


⚠ Job 18601067 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601067, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601067/18601067_submitted.pkl
slurmstepd: error: *** JOB 18601067 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***


⚠ Job 18601068 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601068, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601068/18601068_submitted.pkl
slurmstepd: error: *** JOB 18601068 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***


⚠ Job 18601061 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,123) - Starting with JobEnvironment(job_id=18601061, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,125) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601061/18601061_submitted.pkl
slurmstepd: error: *** JOB 18601061 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***


⚠ Job 18601060 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:17,966) - Starting with JobEnvironment(job_id=18601060, hostname=n1347, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:17,968) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601060/18601060_submitted.pkl
slurmstepd: error: *** JOB 18601060 ON n1347 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***

⚠ Job 18601075 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601075, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601075/18601075_submitted.pkl
slurmstepd: error: *** JOB 18601075 ON n1255 CANCELLED AT 2025-07-24T11:56:21 DUE TO TIME LIMIT ***


⚠ Job 18601066 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601066, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601066/18601066_submitted.pkl
slurmstepd: error: *** JOB 18601066 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***


⚠ Job 18601065 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601065, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601065/18601065_submitted.pkl
slurmstepd: error: *** JOB 18601065 ON n1255 CANCELLED AT 2025-07-24T11:54:20 DUE TO TIME LIMIT ***


⚠ Job 18601055 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,457) - Starting with JobEnvironment(job_id=18601055, hostname=n1588, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,459) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601055/18601055_submitted.pkl
slurmstepd: error: *** JOB 18601055 ON n1588 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***



⚠ Job 18601050 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,290) - Starting with JobEnvironment(job_id=18601050, hostname=n1513, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,293) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601050/18601050_submitted.pkl
slurmstepd: error: *** JOB 18601050 ON n1513 CANCELLED AT 2025-07-24T11:53:20 DUE TO TIME LIMIT ***


⚠ Job 18601059 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:17,965) - Starting with JobEnvironment(job_id=18601059, hostname=n1347, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:17,967) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601059/18601059_submitted.pkl
slurmstepd: error: *** JOB 18601059 ON n1347 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***

⚠ Job 18601063 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,124) - Starting with JobEnvironment(job_id=18601063, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,126) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601063/18601063_submitted.pkl
slurmstepd: error: *** JOB 18601063 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***


⚠ Job 18601062 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,124) - Starting with JobEnvironment(job_id=18601062, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,126) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601062/18601062_submitted.pkl
slurmstepd: error: *** JOB 18601062 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***


⚠ Job 18601064 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:11,123) - Starting with JobEnvironment(job_id=18601064, hostname=n1331, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:11,125) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601064/18601064_submitted.pkl
slurmstepd: error: *** JOB 18601064 ON n1331 CANCELLED AT 2025-07-24T11:53:50 DUE TO TIME LIMIT ***


BoTorchGenerator, failed: 322 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             :  20%|██░░░░░░░░| 100/500 [22:31:11<299:40:38, 2697.10s/it]
⚠ Job 18601096 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601096, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601096/18601096_submitted.pkl
slurmstepd: error: *** JOB 18601096 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


⚠ Job 18601079 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:13,935) - Starting with JobEnvironment(job_id=18601079, hostname=n1490, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:13,937) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601079/18601079_submitted.pkl
slurmstepd: error: *** JOB 18601079 ON n1490 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


⚠ Job 18601092 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601092, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601092/18601092_submitted.pkl
slurmstepd: error: *** JOB 18601092 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


⚠ Job 18601106 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601106, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601106/18601106_submitted.pkl
slurmstepd: error: *** JOB 18601106 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


⚠ Job 18601091 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601091, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601091/18601091_submitted.pkl
slurmstepd: error: *** JOB 18601091 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


⚠ Job 18601090 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601090, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601090/18601090_submitted.pkl
slurmstepd: error: *** JOB 18601090 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***

⚠ Job 18601105 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601105, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601105/18601105_submitted.pkl
slurmstepd: error: *** JOB 18601105 ON n1255 CANCELLED AT 2025-07-24T12:02:52 DUE TO TIME LIMIT ***



⚠ Job 18601104 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601104, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,056) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601104/18601104_submitted.pkl
slurmstepd: error: *** JOB 18601104 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***

⚠ Job 18601102 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:03:12,055) - Starting with JobEnvironment(job_id=18601102, hostname=n1255, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:03:12,057) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601102/18601102_submitted.pkl
slurmstepd: error: *** JOB 18601102 ON n1255 CANCELLED AT 2025-07-24T12:02:51 DUE TO TIME LIMIT ***


BoTorchGenerator, failed: 331 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                :  20%|██░░░░░░░░| 100/500 [22:39:38<299:40:38, 2697.10s/it]
⚠ Job 18601109 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601109, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601109/18601109_submitted.pkl
slurmstepd: error: *** JOB 18601109 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***


⚠ Job 18601111 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601111, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601111/18601111_submitted.pkl
slurmstepd: error: *** JOB 18601111 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***


⚠ Job 18601110 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601110, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601110/18601110_submitted.pkl
slurmstepd: error: *** JOB 18601110 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***


⚠ Job 18601121 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,957) - Starting with JobEnvironment(job_id=18601121, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,959) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601121/18601121_submitted.pkl
slurmstepd: error: *** JOB 18601121 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***


⚠ Job 18601107 (task: 0) with path /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_0_result.pkl
has not produced any output (state: TIMEOUT)
No error stream produced. Look at stdout: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_0_log.out
----------------------------------------
submitit INFO (2025-07-24 11:11:14,988) - Starting with JobEnvironment(job_id=18601107, hostname=n1528, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-07-24 11:11:14,990) - Loading pickle: /data/horse/ws/pwinkler-mnist_tst/omniopt/runs/mnist_cpu/2/single_runs/18601107/18601107_submitted.pkl
slurmstepd: error: *** JOB 18601107 ON n1528 CANCELLED AT 2025-07-24T12:11:28 DUE TO TIME LIMIT ***

BoTorchGenerator, failed: 336 ('VAL_ACC: ' not found), best VAL_ACC: 98.47, eval #30/50 start                                                                    :  20%|██░░░░░░░░| 100/500 [23:59:37<299:40:38, 2697.10s/it]

Copy raw data to clipboard

Download »outfile« as file

Copy raw data to clipboard

Download »progressbar« as file

2025-07-23 13:32:23: SOBOL, Started OmniOpt2 run...
2025-07-23 13:32:33: Sobol, getting new HP set #1/50
2025-07-23 13:32:41: Sobol, getting new HP set #2/50
2025-07-23 13:32:50: Sobol, getting new HP set #3/50
2025-07-23 13:33:00: Sobol, getting new HP set #4/50
2025-07-23 13:33:09: Sobol, getting new HP set #5/50
2025-07-23 13:33:19: Sobol, getting new HP set #6/50
2025-07-23 13:33:29: Sobol, getting new HP set #7/50
2025-07-23 13:33:39: Sobol, getting new HP set #8/50
2025-07-23 13:33:50: Sobol, getting new HP set #9/50
2025-07-23 13:34:02: Sobol, getting new HP set #10/50
2025-07-23 13:34:15: Sobol, getting new HP set #11/50
2025-07-23 13:34:25: Sobol, getting new HP set #12/50
2025-07-23 13:34:35: Sobol, getting new HP set #13/50
2025-07-23 13:34:47: Sobol, getting new HP set #14/50
2025-07-23 13:34:58: Sobol, getting new HP set #15/50
2025-07-23 13:35:10: Sobol, getting new HP set #16/50
2025-07-23 13:35:21: Sobol, getting new HP set #17/50
2025-07-23 13:35:33: Sobol, getting new HP set #18/50
2025-07-23 13:35:46: Sobol, getting new HP set #19/50
2025-07-23 13:35:57: Sobol, getting new HP set #20/50
2025-07-23 13:36:08: Sobol, getting new HP set #21/50
2025-07-23 13:36:18: Sobol, getting new HP set #22/50
2025-07-23 13:36:29: Sobol, getting new HP set #23/50
2025-07-23 13:36:37: Sobol, getting new HP set #24/50
2025-07-23 13:36:45: Sobol, getting new HP set #25/50
2025-07-23 13:36:54: Sobol, getting new HP set #26/50
2025-07-23 13:37:02: Sobol, getting new HP set #27/50
2025-07-23 13:37:11: Sobol, getting new HP set #28/50
2025-07-23 13:37:19: Sobol, getting new HP set #29/50
2025-07-23 13:37:28: Sobol, getting new HP set #30/50
2025-07-23 13:37:36: Sobol, getting new HP set #31/50
2025-07-23 13:37:44: Sobol, getting new HP set #32/50
2025-07-23 13:37:53: Sobol, getting new HP set #33/50
2025-07-23 13:38:01: Sobol, getting new HP set #34/50
2025-07-23 13:38:10: Sobol, getting new HP set #35/50
2025-07-23 13:38:18: Sobol, getting new HP set #36/50
2025-07-23 13:38:26: Sobol, getting new HP set #37/50
2025-07-23 13:38:36: Sobol, getting new HP set #38/50
2025-07-23 13:38:44: Sobol, getting new HP set #39/50
2025-07-23 13:38:52: Sobol, getting new HP set #40/50
2025-07-23 13:39:00: Sobol, getting new HP set #41/50
2025-07-23 13:39:08: Sobol, getting new HP set #42/50
2025-07-23 13:39:17: Sobol, getting new HP set #43/50
2025-07-23 13:39:25: Sobol, getting new HP set #44/50
2025-07-23 13:39:33: Sobol, getting new HP set #45/50
2025-07-23 13:39:42: Sobol, getting new HP set #46/50
2025-07-23 13:39:50: Sobol, getting new HP set #47/50
2025-07-23 13:39:59: Sobol, getting new HP set #48/50
2025-07-23 13:40:08: Sobol, getting new HP set #49/50
2025-07-23 13:40:16: Sobol, getting new HP set #50/50
2025-07-23 13:40:24: Sobol, requested 50 jobs, got 50, 9.45 s/job
2025-07-23 13:40:28: Sobol, eval #1/50 start                     
2025-07-23 13:40:33: Sobol, eval #2/50 start                     
2025-07-23 13:40:39: Sobol, eval #3/50 start                     
2025-07-23 13:40:44: Sobol, eval #4/50 start                     
2025-07-23 13:40:49: Sobol, eval #5/50 start                     
2025-07-23 13:40:53: Sobol, eval #6/50 start                     
2025-07-23 13:40:58: Sobol, eval #7/50 start                     
2025-07-23 13:41:02: Sobol, eval #8/50 start                     
2025-07-23 13:41:07: Sobol, eval #9/50 start                     
2025-07-23 13:41:11: Sobol, eval #10/50 start                    
2025-07-23 13:41:16: Sobol, eval #11/50 start                    
2025-07-23 13:41:20: Sobol, eval #12/50 start                    
2025-07-23 13:41:25: Sobol, eval #13/50 start                    
2025-07-23 13:41:29: Sobol, eval #14/50 start                    
2025-07-23 13:41:35: Sobol, eval #15/50 start                    
2025-07-23 13:41:39: Sobol, eval #16/50 start                    
2025-07-23 13:41:44: Sobol, eval #17/50 start                    
2025-07-23 13:41:48: Sobol, eval #18/50 start                    
2025-07-23 13:41:52: Sobol, eval #19/50 start                    
2025-07-23 13:41:57: Sobol, eval #20/50 start                    
2025-07-23 13:42:02: Sobol, eval #21/50 start                    
2025-07-23 13:42:06: Sobol, eval #22/50 start                    
2025-07-23 13:42:11: Sobol, eval #23/50 start                    
2025-07-23 13:42:15: Sobol, eval #24/50 start                    
2025-07-23 13:42:20: Sobol, eval #25/50 start                    
2025-07-23 13:42:24: Sobol, eval #26/50 start                    
2025-07-23 13:42:28: Sobol, eval #27/50 start                    
2025-07-23 13:42:34: Sobol, eval #28/50 start                    
2025-07-23 13:42:39: Sobol, eval #29/50 start                    
2025-07-23 13:42:43: Sobol, eval #30/50 start                    
2025-07-23 13:42:47: Sobol, eval #31/50 start                    
2025-07-23 13:42:52: Sobol, eval #32/50 start                    
2025-07-23 13:42:56: Sobol, eval #33/50 start                    
2025-07-23 13:43:01: Sobol, eval #34/50 start                    
2025-07-23 13:43:05: Sobol, eval #35/50 start                    
2025-07-23 13:43:10: Sobol, eval #36/50 start                    
2025-07-23 13:43:14: Sobol, eval #37/50 start                    
2025-07-23 13:43:18: Sobol, eval #38/50 start                    
2025-07-23 13:43:23: Sobol, eval #39/50 start                    
2025-07-23 13:43:27: Sobol, eval #40/50 start                    
2025-07-23 13:43:32: Sobol, eval #41/50 start                    
2025-07-23 13:43:36: Sobol, eval #42/50 start                    
2025-07-23 13:43:40: Sobol, eval #43/50 start                    
2025-07-23 13:43:45: Sobol, eval #44/50 start                    
2025-07-23 13:43:49: Sobol, eval #45/50 start                    
2025-07-23 13:43:54: Sobol, eval #46/50 start                    
2025-07-23 13:43:58: Sobol, eval #47/50 start                    
2025-07-23 13:44:03: Sobol, eval #48/50 start                    
2025-07-23 13:44:07: Sobol, eval #49/50 start                    
2025-07-23 13:44:12: Sobol, eval #50/50 start                    
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:18: Sobol, starting new job                     
2025-07-23 13:44:46: Sobol, unknown 8 = ∑8/50, started new job   
2025-07-23 13:44:46: Sobol, unknown 8 = ∑8/50, started new job   
2025-07-23 13:44:46: Sobol, unknown 8 = ∑8/50, started new job   
2025-07-23 13:44:46: Sobol, unknown 9 = ∑9/50, started new job   
2025-07-23 13:44:46: Sobol, unknown 7 = ∑7/50, started new job   
2025-07-23 13:44:47: Sobol, unknown 10 = ∑10/50, started new job 
2025-07-23 13:44:47: Sobol, unknown 11 = ∑11/50, started new job 
2025-07-23 13:44:47: Sobol, unknown 11 = ∑11/50, started new job 
2025-07-23 13:44:47: Sobol, unknown 14 = ∑14/50, started new job 
2025-07-23 13:44:48: Sobol, pending/unknown 12/2 = ∑14/50, started new job
2025-07-23 13:44:49: Sobol, pending/unknown 13/3 = ∑16/50, started new job
2025-07-23 13:44:50: Sobol, pending/unknown 13/3 = ∑16/50, started new job
2025-07-23 13:44:51: Sobol, pending/unknown 14/2 = ∑16/50, started new job
2025-07-23 13:44:51: Sobol, pending/unknown 14/2 = ∑16/50, started new job
2025-07-23 13:44:52: Sobol, pending/unknown 14/2 = ∑16/50, started new job
2025-07-23 13:44:52: Sobol, pending/unknown 14/2 = ∑16/50, started new job
2025-07-23 13:45:26: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:26: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:26: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:28: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:29: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:29: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:29: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:29: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:31: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:31: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:31: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:32: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:32: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:45:33: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:46:00: Sobol, running/pending 10/6 = ∑16/50, starting new job
2025-07-23 13:46:05: Sobol, running/unknown 16/1 = ∑17/50, starting new job
2025-07-23 13:46:06: Sobol, running/unknown 16/1 = ∑17/50, started new job 
2025-07-23 13:46:07: Sobol, running/unknown 16/3 = ∑19/50, started new job 
2025-07-23 13:46:08: Sobol, running/unknown 16/3 = ∑19/50, started new job 
2025-07-23 13:46:13: Sobol, running/pending/unknown 16/3/1 = ∑20/50, started new job
2025-07-23 13:46:14: Sobol, running/pending/unknown 16/6/1 = ∑23/50, started new job
2025-07-23 13:46:17: Sobol, running/pending/unknown 16/6/3 = ∑25/50, started new job
2025-07-23 13:46:17: Sobol, running/pending/unknown 16/6/3 = ∑25/50, started new job
2025-07-23 13:46:18: Sobol, running/pending/unknown 16/6/6 = ∑28/50, started new job
2025-07-23 13:46:19: Sobol, running/pending/unknown 16/6/8 = ∑30/50, started new job
2025-07-23 13:46:19: Sobol, running/pending/unknown 16/6/7 = ∑29/50, started new job
2025-07-23 13:46:19: Sobol, running/pending/unknown 16/6/8 = ∑30/50, started new job
2025-07-23 13:46:19: Sobol, running/pending/unknown 16/6/8 = ∑30/50, started new job
2025-07-23 13:46:21: Sobol, running/pending/unknown 16/8/6 = ∑30/50, started new job
2025-07-23 13:46:21: Sobol, running/pending/unknown 16/8/6 = ∑30/50, started new job
2025-07-23 13:46:24: Sobol, running/pending/unknown 16/14/1 = ∑31/50, started new job
2025-07-23 13:46:40: Sobol, running/pending/unknown 26/5/1 = ∑32/50, started new job 
2025-07-23 13:46:41: Sobol, running/pending 26/6 = ∑32/50, starting new job          
2025-07-23 13:46:43: Sobol, running/pending 26/6 = ∑32/50, starting new job          
2025-07-23 13:46:45: Sobol, running/pending 26/6 = ∑32/50, starting new job          
2025-07-23 13:46:56: Sobol, running/pending 26/6 = ∑32/50, starting new job          
2025-07-23 13:46:56: Sobol, running/pending 26/6 = ∑32/50, starting new job          
2025-07-23 13:47:21: Sobol, running 32 = ∑32/50, starting new job                    
2025-07-23 13:47:22: Sobol, running 32 = ∑32/50, starting new job                    
2025-07-23 13:47:22: Sobol, running 32 = ∑32/50, starting new job                    
2025-07-23 13:47:22: Sobol, running/pending/unknown 32/1/1 = ∑34/50, starting new job
2025-07-23 13:47:22: Sobol, running/pending/unknown 32/1/1 = ∑34/50, starting new job
2025-07-23 13:47:22: Sobol, running/pending/unknown 32/1/2 = ∑35/50, starting new job
2025-07-23 13:47:24: Sobol, running/pending/unknown 32/1/4 = ∑37/50, starting new job
2025-07-23 13:47:24: Sobol, running/pending/unknown 32/1/4 = ∑37/50, starting new job
2025-07-23 13:47:24: Sobol, running/pending/unknown 32/1/4 = ∑37/50, starting new job
2025-07-23 13:47:24: Sobol, running/pending/unknown 32/1/4 = ∑37/50, starting new job
2025-07-23 13:47:25: Sobol, running/pending/unknown 32/1/4 = ∑37/50, started new job 
2025-07-23 13:47:26: Sobol, running/pending/unknown 32/1/4 = ∑37/50, started new job 
2025-07-23 13:47:26: Sobol, running/pending/unknown 32/1/4 = ∑37/50, started new job 
2025-07-23 13:47:26: Sobol, running/pending/unknown 32/1/4 = ∑37/50, started new job 
2025-07-23 13:47:26: Sobol, running/pending/unknown 32/1/4 = ∑37/50, started new job 
2025-07-23 13:47:27: Sobol, running/pending/unknown 32/1/4 = ∑37/50, starting new job
2025-07-23 13:48:09: Sobol, running/unknown 37/1 = ∑38/50, started new job           
2025-07-23 13:48:12: Sobol, running/pending/unknown 37/1/2 = ∑40/50, starting new job
2025-07-23 13:48:13: Sobol, running/pending/unknown 37/1/2 = ∑40/50, started new job 
2025-07-23 13:48:13: Sobol, running/pending/unknown 37/1/2 = ∑40/50, started new job 
2025-07-23 13:48:20: Sobol, running/pending/unknown 37/3/1 = ∑41/50, started new job 
2025-07-23 13:48:28: Sobol, running/pending/unknown 37/4/7 = ∑48/50, started new job 
2025-07-23 13:48:28: Sobol, running/pending 37/11 = ∑48/50, started new job          
2025-07-23 13:48:28: Sobol, running/pending 37/11 = ∑48/50, started new job          
2025-07-23 13:48:28: Sobol, running/pending 37/11 = ∑48/50, started new job          
2025-07-23 13:48:28: Sobol, running/pending 37/11 = ∑48/50, started new job          
2025-07-23 13:48:28: Sobol, running/pending/unknown 37/4/7 = ∑48/50, started new job 
2025-07-23 13:48:28: Sobol, running/pending 37/11 = ∑48/50, started new job          
2025-07-23 13:48:29: Sobol, running/pending 37/11 = ∑48/50, starting new job         
2025-07-23 13:48:33: Sobol, running/pending/unknown 37/11/1 = ∑49/50, started new job
2025-07-23 13:49:14: Sobol, running/unknown 49/1 = ∑50/50, started new job           
2025-07-23 13:49:24: Sobol, running/pending 49/1 = ∑50/50, waiting for 50 jobs       
2025-07-23 13:49:36: Sobol, running/pending 49/1 = ∑50/50, waiting for 50 jobs       
2025-07-23 13:49:48: Sobol, running/pending 49/1 = ∑50/50, waiting for 50 jobs       
2025-07-23 13:50:00: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:50:12: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:50:29: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:50:41: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:50:53: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:51:05: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:51:17: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:51:29: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:51:41: Sobol, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 13:51:53: Sobol, running 50 = ∑50/50, new result: 11.35                   
2025-07-23 13:52:15: Sobol, best VAL_ACC: 11.35, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job
2025-07-23 13:52:21: Sobol, best VAL_ACC: 11.35, running 49 = ∑49/50, waiting for 49 jobs                
2025-07-23 13:52:33: Sobol, best VAL_ACC: 11.35, running 49 = ∑49/50, waiting for 49 jobs                
2025-07-23 13:52:45: Sobol, best VAL_ACC: 11.35, running 49 = ∑49/50, new result: 10.14                  
2025-07-23 13:53:06: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job
2025-07-23 13:53:12: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:53:24: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:53:36: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:53:48: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:54:00: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:54:13: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:54:25: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:54:37: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:54:49: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:55:02: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:55:14: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:55:26: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:55:38: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:55:51: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:56:03: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:56:17: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:56:29: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:56:41: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:56:55: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:57:08: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:57:21: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:57:33: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:57:46: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:57:58: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:58:11: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:58:23: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:58:36: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:58:49: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:59:03: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:59:15: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:59:33: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:59:45: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 13:59:57: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:00:10: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:00:22: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:00:34: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:00:46: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:01:00: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, waiting for 48 jobs                
2025-07-23 14:01:12: Sobol, best VAL_ACC: 11.35, running 48 = ∑48/50, new result: 9.58                   
2025-07-23 14:01:38: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job
2025-07-23 14:01:45: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:01:57: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:02:09: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:02:22: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:02:34: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:02:46: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:02:58: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:03:11: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:03:23: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:03:35: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:03:48: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:04:00: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:04:12: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:04:25: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:04:41: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:04:53: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:05:11: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:05:23: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:05:38: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:05:51: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:06:03: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:06:16: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:06:30: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:06:43: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:06:58: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:07:11: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:07:24: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:07:37: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:07:50: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:08:03: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:08:15: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:08:28: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:08:40: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:08:53: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:09:05: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:09:17: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:09:29: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:09:42: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:10:06: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:10:18: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:10:30: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:10:49: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:11:04: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:11:17: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:11:29: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:11:41: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:11:55: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:12:08: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:12:20: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:12:32: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:12:49: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:13:03: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:13:15: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:13:27: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:13:41: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:13:53: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:14:05: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:14:17: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, waiting for 47 jobs                
2025-07-23 14:14:32: Sobol, best VAL_ACC: 11.35, running 47 = ∑47/50, new result: 8.83                   
2025-07-23 14:14:54: Sobol, best VAL_ACC: 11.35, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-23 14:15:02: Sobol, best VAL_ACC: 11.35, running 46 = ∑46/50, waiting for 46 jobs                
2025-07-23 14:15:14: Sobol, best VAL_ACC: 11.35, running 46 = ∑46/50, waiting for 46 jobs                
2025-07-23 14:15:27: Sobol, best VAL_ACC: 11.35, running 46 = ∑46/50, waiting for 46 jobs                
2025-07-23 14:15:39: Sobol, best VAL_ACC: 11.35, running 46 = ∑46/50, new result: 11.34                  
2025-07-23 14:16:02: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job
2025-07-23 14:16:09: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:16:22: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:16:34: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:16:47: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:17:00: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:17:21: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:17:34: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:17:46: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:18:05: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:18:20: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:18:32: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:18:44: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:18:56: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:19:09: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:19:22: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, waiting for 45 jobs                
2025-07-23 14:19:34: Sobol, best VAL_ACC: 11.35, running 45 = ∑45/50, new result: 11.35                  
2025-07-23 14:19:55: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job
2025-07-23 14:20:02: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 44 jobs                
2025-07-23 14:20:14: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 44 jobs                
2025-07-23 14:20:33: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 44 jobs                
2025-07-23 14:20:45: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 44 jobs                
2025-07-23 14:20:58: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, waiting for 44 jobs                
2025-07-23 14:21:10: Sobol, best VAL_ACC: 11.35, running 44 = ∑44/50, new result: 9.82                   
2025-07-23 14:21:36: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 44 jobs, finished 1 job
2025-07-23 14:21:43: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:21:55: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:22:09: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:22:21: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:22:35: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:22:47: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:22:59: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:23:12: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:23:24: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:23:37: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:23:49: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:24:01: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:24:13: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:24:26: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:24:39: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:24:51: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:25:04: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:25:17: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:25:30: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:25:42: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:25:54: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:26:06: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:26:19: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:26:33: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:26:46: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, waiting for 43 jobs                
2025-07-23 14:26:59: Sobol, best VAL_ACC: 11.35, running 43 = ∑43/50, new result: 11.35                  
2025-07-23 14:27:19: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 43 jobs, finished 1 job
2025-07-23 14:27:26: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:27:38: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:27:51: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:28:03: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:28:16: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:28:28: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:28:41: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:28:53: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:29:06: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:29:19: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:29:32: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:29:52: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:30:05: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:30:18: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:30:31: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:30:43: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:31:03: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:31:17: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:31:39: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:31:54: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:32:08: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:32:22: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:32:35: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:32:53: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:33:07: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:33:19: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:33:32: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:33:47: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:34:00: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:34:13: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:34:26: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:34:40: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:34:52: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:35:08: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:35:20: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, waiting for 42 jobs                
2025-07-23 14:35:33: Sobol, best VAL_ACC: 11.35, running 42 = ∑42/50, new result: 17.57                  
2025-07-23 14:35:54: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 42 jobs, finished 1 job
2025-07-23 14:36:01: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 41 jobs                
2025-07-23 14:36:14: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 41 jobs                
2025-07-23 14:36:27: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 41 jobs                
2025-07-23 14:36:40: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 41 jobs                
2025-07-23 14:36:52: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, waiting for 41 jobs                
2025-07-23 14:37:06: Sobol, best VAL_ACC: 17.57, running 41 = ∑41/50, new result: 8.36                   
2025-07-23 14:37:27: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 41 jobs, finished 1 job
2025-07-23 14:37:35: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:37:48: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:38:00: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:38:12: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:38:25: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:38:37: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:38:50: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:39:03: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:39:15: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:39:30: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:39:42: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, waiting for 40 jobs                
2025-07-23 14:39:54: Sobol, best VAL_ACC: 17.57, running 40 = ∑40/50, new result: 21.99                  
2025-07-23 14:40:24: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 40 jobs, finished 1 job
2025-07-23 14:40:31: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                
2025-07-23 14:40:43: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                
2025-07-23 14:40:56: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                
2025-07-23 14:41:08: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                
2025-07-23 14:41:21: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                
2025-07-23 14:41:34: Sobol, best VAL_ACC: 21.99, running 39 = ∑39/50, new result: 15.38                  
2025-07-23 14:41:55: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job
2025-07-23 14:42:05: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:42:18: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:42:31: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:42:50: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:43:06: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:43:18: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:43:31: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:43:45: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:43:58: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:44:12: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:44:27: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:44:43: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                
2025-07-23 14:44:56: Sobol, best VAL_ACC: 21.99, running 38 = ∑38/50, new result: 9.8                    
2025-07-23 14:45:25: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job
2025-07-23 14:45:32: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:45:48: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:46:10: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:46:25: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:46:38: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:46:53: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:47:06: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:47:21: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:47:36: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:47:58: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:48:12: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:48:24: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:48:37: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:48:50: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:50:15: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:50:32: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:50:45: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:50:58: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:51:11: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:51:25: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:51:49: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:52:10: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:52:24: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:52:37: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:52:50: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:53:03: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:53:18: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:53:30: Sobol, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                
2025-07-23 14:54:03: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:03: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:03: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:04: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:06: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:06: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:06: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:07: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:07: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:05: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:07: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:07: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:07: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:08: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:08: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:54:10: Sobol, best VAL_ACC: 21.99, timeout 37 = ∑37/50, job_failed                         
2025-07-23 14:55:09: Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     
2025-07-23 14:55:13: Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     
2025-07-23 14:55:13: Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     
2025-07-23 14:55:13: Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     
2025-07-23 14:55:13: Sobol, failed: 5, best VAL_ACC: 21.99, timeout 32 = ∑32/50, job_failed     
2025-07-23 14:58:14: Sobol, failed: 37, best VAL_ACC: 21.99, waiting for 37 jobs, finished 37 jobs
2025-07-23 14:58:23: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #1/50             
2025-07-23 14:58:37: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #2/50             
2025-07-23 14:58:51: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #3/50             
2025-07-23 14:59:05: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #4/50             
2025-07-23 14:59:18: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #5/50             
2025-07-23 14:59:32: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #6/50             
2025-07-23 14:59:45: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #7/50             
2025-07-23 14:59:59: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #8/50             
2025-07-23 15:00:13: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #9/50             
2025-07-23 15:00:26: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #10/50            
2025-07-23 15:00:41: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #11/50            
2025-07-23 15:00:54: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #12/50            
2025-07-23 15:01:08: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #13/50            
2025-07-23 15:01:22: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #14/50            
2025-07-23 15:01:46: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #15/50            
2025-07-23 15:02:00: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #16/50            
2025-07-23 15:02:13: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #17/50            
2025-07-23 15:02:26: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #18/50            
2025-07-23 15:02:45: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #19/50            
2025-07-23 15:02:58: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #20/50            
2025-07-23 15:03:11: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #21/50            
2025-07-23 15:03:27: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #22/50            
2025-07-23 15:03:42: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #23/50            
2025-07-23 15:03:55: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #24/50            
2025-07-23 15:04:09: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #25/50            
2025-07-23 15:04:24: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #26/50            
2025-07-23 15:04:39: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #27/50            
2025-07-23 15:04:53: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #28/50            
2025-07-23 15:05:08: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #29/50            
2025-07-23 15:05:21: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #30/50            
2025-07-23 15:05:36: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #31/50            
2025-07-23 15:05:51: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #32/50            
2025-07-23 15:06:12: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #33/50            
2025-07-23 15:06:26: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #34/50            
2025-07-23 15:06:39: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #35/50            
2025-07-23 15:06:53: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #36/50            
2025-07-23 15:07:07: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #37/50            
2025-07-23 15:07:21: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #38/50            
2025-07-23 15:07:37: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #39/50            
2025-07-23 15:07:50: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #40/50            
2025-07-23 15:08:05: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #41/50            
2025-07-23 15:08:18: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #42/50            
2025-07-23 15:08:36: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #43/50            
2025-07-23 15:08:51: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #44/50            
2025-07-23 15:09:04: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #45/50            
2025-07-23 15:09:18: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #46/50            
2025-07-23 15:09:36: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #47/50            
2025-07-23 15:09:51: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #48/50            
2025-07-23 15:10:04: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #49/50            
2025-07-23 15:10:19: Sobol, failed: 37, best VAL_ACC: 21.99, getting new HP set #50/50            
2025-07-23 15:10:33: Sobol, failed: 37, best VAL_ACC: 21.99, requested 50 jobs, got 50, 14.63 s/job
2025-07-23 15:10:39: Sobol, failed: 37, best VAL_ACC: 21.99, eval #1/50 start                      
2025-07-23 15:10:47: Sobol, failed: 37, best VAL_ACC: 21.99, eval #2/50 start                      
2025-07-23 15:10:54: Sobol, failed: 37, best VAL_ACC: 21.99, eval #3/50 start                      
2025-07-23 15:11:01: Sobol, failed: 37, best VAL_ACC: 21.99, eval #4/50 start                      
2025-07-23 15:11:09: Sobol, failed: 37, best VAL_ACC: 21.99, eval #5/50 start                      
2025-07-23 15:11:16: Sobol, failed: 37, best VAL_ACC: 21.99, eval #6/50 start                      
2025-07-23 15:11:26: Sobol, failed: 37, best VAL_ACC: 21.99, eval #7/50 start                      
2025-07-23 15:11:33: Sobol, failed: 37, best VAL_ACC: 21.99, eval #8/50 start                      
2025-07-23 15:11:41: Sobol, failed: 37, best VAL_ACC: 21.99, eval #9/50 start                      
2025-07-23 15:11:48: Sobol, failed: 37, best VAL_ACC: 21.99, eval #10/50 start                     
2025-07-23 15:11:55: Sobol, failed: 37, best VAL_ACC: 21.99, eval #11/50 start                     
2025-07-23 15:12:03: Sobol, failed: 37, best VAL_ACC: 21.99, eval #12/50 start                     
2025-07-23 15:12:11: Sobol, failed: 37, best VAL_ACC: 21.99, eval #13/50 start                     
2025-07-23 15:12:18: Sobol, failed: 37, best VAL_ACC: 21.99, eval #14/50 start                     
2025-07-23 15:12:26: Sobol, failed: 37, best VAL_ACC: 21.99, eval #15/50 start                     
2025-07-23 15:12:34: Sobol, failed: 37, best VAL_ACC: 21.99, eval #16/50 start                     
2025-07-23 15:12:41: Sobol, failed: 37, best VAL_ACC: 21.99, eval #17/50 start                     
2025-07-23 15:12:52: Sobol, failed: 37, best VAL_ACC: 21.99, eval #18/50 start                     
2025-07-23 15:13:05: Sobol, failed: 37, best VAL_ACC: 21.99, eval #19/50 start                     
2025-07-23 15:13:13: Sobol, failed: 37, best VAL_ACC: 21.99, eval #20/50 start                     
2025-07-23 15:13:20: Sobol, failed: 37, best VAL_ACC: 21.99, eval #21/50 start                     
2025-07-23 15:13:28: Sobol, failed: 37, best VAL_ACC: 21.99, eval #22/50 start                     
2025-07-23 15:13:35: Sobol, failed: 37, best VAL_ACC: 21.99, eval #23/50 start                     
2025-07-23 15:13:42: Sobol, failed: 37, best VAL_ACC: 21.99, eval #24/50 start                     
2025-07-23 15:13:49: Sobol, failed: 37, best VAL_ACC: 21.99, eval #25/50 start                     
2025-07-23 15:13:57: Sobol, failed: 37, best VAL_ACC: 21.99, eval #26/50 start                     
2025-07-23 15:14:05: Sobol, failed: 37, best VAL_ACC: 21.99, eval #27/50 start                     
2025-07-23 15:14:12: Sobol, failed: 37, best VAL_ACC: 21.99, eval #28/50 start                     
2025-07-23 15:14:19: Sobol, failed: 37, best VAL_ACC: 21.99, eval #29/50 start                     
2025-07-23 15:14:27: Sobol, failed: 37, best VAL_ACC: 21.99, eval #30/50 start                     
2025-07-23 15:14:39: Sobol, failed: 37, best VAL_ACC: 21.99, eval #31/50 start                     
2025-07-23 15:14:49: Sobol, failed: 37, best VAL_ACC: 21.99, eval #32/50 start                     
2025-07-23 15:14:57: Sobol, failed: 37, best VAL_ACC: 21.99, eval #33/50 start                     
2025-07-23 15:15:05: Sobol, failed: 37, best VAL_ACC: 21.99, eval #34/50 start                     
2025-07-23 15:15:12: Sobol, failed: 37, best VAL_ACC: 21.99, eval #35/50 start                     
2025-07-23 15:15:20: Sobol, failed: 37, best VAL_ACC: 21.99, eval #36/50 start                     
2025-07-23 15:15:27: Sobol, failed: 37, best VAL_ACC: 21.99, eval #37/50 start                     
2025-07-23 15:15:35: Sobol, failed: 37, best VAL_ACC: 21.99, eval #38/50 start                     
2025-07-23 15:15:42: Sobol, failed: 37, best VAL_ACC: 21.99, eval #39/50 start                     
2025-07-23 15:15:49: Sobol, failed: 37, best VAL_ACC: 21.99, eval #40/50 start                     
2025-07-23 15:15:57: Sobol, failed: 37, best VAL_ACC: 21.99, eval #41/50 start                     
2025-07-23 15:16:06: Sobol, failed: 37, best VAL_ACC: 21.99, eval #42/50 start                     
2025-07-23 15:16:13: Sobol, failed: 37, best VAL_ACC: 21.99, eval #43/50 start                     
2025-07-23 15:16:21: Sobol, failed: 37, best VAL_ACC: 21.99, eval #44/50 start                     
2025-07-23 15:16:39: Sobol, failed: 37, best VAL_ACC: 21.99, eval #45/50 start                     
2025-07-23 15:16:48: Sobol, failed: 37, best VAL_ACC: 21.99, eval #46/50 start                     
2025-07-23 15:16:58: Sobol, failed: 37, best VAL_ACC: 21.99, eval #47/50 start                     
2025-07-23 15:17:07: Sobol, failed: 37, best VAL_ACC: 21.99, eval #48/50 start                     
2025-07-23 15:17:15: Sobol, failed: 37, best VAL_ACC: 21.99, eval #49/50 start                     
2025-07-23 15:17:24: Sobol, failed: 37, best VAL_ACC: 21.99, eval #50/50 start                     
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:34: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:35: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:17:35: Sobol, failed: 37, best VAL_ACC: 21.99, starting new job                      
2025-07-23 15:18:48: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 6 = ∑6/50, started new job    
2025-07-23 15:18:48: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 7 = ∑7/50, started new job    
2025-07-23 15:18:49: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 12 = ∑12/50, started new job  
2025-07-23 15:18:49: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 12 = ∑12/50, started new job  
2025-07-23 15:18:49: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 12 = ∑12/50, started new job  
2025-07-23 15:18:50: Sobol, failed: 37, best VAL_ACC: 21.99, unknown 12 = ∑12/50, started new job  
2025-07-23 15:18:50: Sobol, failed: 37, best VAL_ACC: 21.99, pending 13 = ∑13/50, started new job  
2025-07-23 15:18:51: Sobol, failed: 37, best VAL_ACC: 21.99, pending 13 = ∑13/50, started new job  
2025-07-23 15:18:51: Sobol, failed: 37, best VAL_ACC: 21.99, pending 13 = ∑13/50, started new job  
2025-07-23 15:18:51: Sobol, failed: 37, best VAL_ACC: 21.99, pending 13 = ∑13/50, started new job  
2025-07-23 15:18:52: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:18:52: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:18:54: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:18:55: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:18:55: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:18:55: Sobol, failed: 37, best VAL_ACC: 21.99, pending 16 = ∑16/50, started new job  
2025-07-23 15:20:16: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:17: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:22: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:20:22: Sobol, failed: 37, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job 
2025-07-23 15:21:27: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 16/1 = ∑17/50, started new job
2025-07-23 15:21:35: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, started new job
2025-07-23 15:21:35: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 15:21:38: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 16/2/3 = ∑21/50, started new job 
2025-07-23 15:21:38: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 16/2/3 = ∑21/50, started new job 
2025-07-23 15:21:38: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 16/2/3 = ∑21/50, starting new job
2025-07-23 15:21:40: Sobol, failed: 37, best VAL_ACC: 21.99, running 25 = ∑25/50, started new job                     
2025-07-23 15:21:41: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/4 = ∑29/50, started new job           
2025-07-23 15:21:41: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/4 = ∑29/50, started new job           
2025-07-23 15:21:41: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/4 = ∑29/50, started new job           
2025-07-23 15:21:41: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/4 = ∑29/50, started new job           
2025-07-23 15:21:43: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/5 = ∑30/50, started new job           
2025-07-23 15:21:43: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/5 = ∑30/50, started new job           
2025-07-23 15:21:43: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/5 = ∑30/50, started new job           
2025-07-23 15:21:45: Sobol, failed: 37, best VAL_ACC: 21.99, running/unknown 25/5 = ∑30/50, started new job           
2025-07-23 15:21:46: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 25/5 = ∑30/50, started new job           
2025-07-23 15:21:49: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 25/5 = ∑30/50, starting new job          
2025-07-23 15:22:48: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 30/1 = ∑31/50, starting new job          
2025-07-23 15:22:48: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 30/1 = ∑31/50, started new job           
2025-07-23 15:22:56: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 30/1/1 = ∑32/50, started new job 
2025-07-23 15:23:00: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 30/2 = ∑32/50, starting new job          
2025-07-23 15:23:00: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 30/2 = ∑32/50, starting new job          
2025-07-23 15:23:02: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 30/2 = ∑32/50, starting new job          
2025-07-23 15:23:15: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, started new job                     
2025-07-23 15:23:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:20: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:20: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:21: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:22: Sobol, failed: 37, best VAL_ACC: 21.99, running 33 = ∑33/50, starting new job                    
2025-07-23 15:23:50: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 33/1 = ∑34/50, started new job           
2025-07-23 15:23:50: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 33/1 = ∑34/50, starting new job          
2025-07-23 15:24:04: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 33/1 = ∑34/50, starting new job          
2025-07-23 15:24:16: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 33/1/2 = ∑36/50, started new job 
2025-07-23 15:24:16: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 33/1/2 = ∑36/50, started new job 
2025-07-23 15:24:24: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 34/2/1 = ∑37/50, started new job 
2025-07-23 15:24:36: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending 34/3 = ∑37/50, starting new job          
2025-07-23 15:24:40: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 34/3/1 = ∑38/50, started new job 
2025-07-23 15:24:43: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 34/4/1 = ∑39/50, started new job 
2025-07-23 15:24:46: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:47: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:47: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:47: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:48: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:48: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:24:50: Sobol, failed: 37, best VAL_ACC: 21.99, running/pending/unknown 39/5/2 = ∑46/50, started new job 
2025-07-23 15:25:06: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed/pending/unknown 38/1/7/1 = ∑47/50, started new job
2025-07-23 15:25:16: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed 47/1 = ∑48/50, started new job                    
2025-07-23 15:25:32: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed 47/1 = ∑48/50, starting new job                   
2025-07-23 15:25:55: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed/unknown 47/1/1 = ∑49/50, started new job          
2025-07-23 15:26:23: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed/pending 48/1/1 = ∑50/50, started new job          
2025-07-23 15:26:33: Sobol, failed: 37, best VAL_ACC: 21.99, running/completed/pending 48/1/1 = ∑50/50, new result: 10.28        
2025-07-23 15:27:07: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, finishing jobs, finished 1 job                 
2025-07-23 15:27:27: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:27:44: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:27:59: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:28:14: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:28:29: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:28:44: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:28:59: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:29:16: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:29:30: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:29:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:30:08: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:30:24: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:30:39: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:30:54: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:31:10: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:31:25: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:31:48: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:32:03: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:32:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                            
2025-07-23 15:32:33: Sobol, failed: 37, best VAL_ACC: 21.99, running 49 = ∑49/50, new result: 13.04                              
2025-07-23 15:33:03: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job            
2025-07-23 15:33:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:33:31: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:33:46: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:34:01: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:34:17: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:34:33: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:34:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:35:05: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:35:33: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:35:52: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:36:09: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:36:30: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:36:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:37:02: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:37:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:37:34: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:37:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:38:08: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:38:26: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:38:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:39:04: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:39:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:39:34: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:39:53: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:40:10: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:40:39: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                            
2025-07-23 15:40:57: Sobol, failed: 37, best VAL_ACC: 21.99, running 48 = ∑48/50, new result: 11.61                              
2025-07-23 15:41:32: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job            
2025-07-23 15:41:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, waiting for 47 jobs                            
2025-07-23 15:42:07: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, waiting for 47 jobs                            
2025-07-23 15:42:24: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, waiting for 47 jobs                            
2025-07-23 15:42:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, waiting for 47 jobs                            
2025-07-23 15:42:57: Sobol, failed: 37, best VAL_ACC: 21.99, running 47 = ∑47/50, new result: 9.74                               
2025-07-23 15:43:31: Sobol, failed: 37, best VAL_ACC: 21.99, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job            
2025-07-23 15:43:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 46 = ∑46/50, waiting for 46 jobs                            
2025-07-23 15:43:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 46 = ∑46/50, new result: 9.81                               
2025-07-23 15:44:27: Sobol, failed: 37, best VAL_ACC: 21.99, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job            
2025-07-23 15:44:39: Sobol, failed: 37, best VAL_ACC: 21.99, running 45 = ∑45/50, waiting for 45 jobs                            
2025-07-23 15:44:57: Sobol, failed: 37, best VAL_ACC: 21.99, running 45 = ∑45/50, waiting for 45 jobs                            
2025-07-23 15:45:13: Sobol, failed: 37, best VAL_ACC: 21.99, running 45 = ∑45/50, new result: 10.43                              
2025-07-23 15:46:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job            
2025-07-23 15:46:28: Sobol, failed: 37, best VAL_ACC: 21.99, running 44 = ∑44/50, waiting for 44 jobs                            
2025-07-23 15:46:43: Sobol, failed: 37, best VAL_ACC: 21.99, running 44 = ∑44/50, new result: 0.34                               
2025-07-23 15:47:33: Sobol, failed: 37, best VAL_ACC: 21.99, running 43 = ∑43/50, waiting for 44 jobs, finished 1 job            
2025-07-23 15:47:42: Sobol, failed: 37, best VAL_ACC: 21.99, running 43 = ∑43/50, waiting for 43 jobs                            
2025-07-23 15:47:58: Sobol, failed: 37, best VAL_ACC: 21.99, running 43 = ∑43/50, new result: 9.82                               
2025-07-23 15:47:58: Sobol, failed: 37, best VAL_ACC: 21.99, running 43 = ∑43/50, new result: 11.35                              
2025-07-23 15:49:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 41 = ∑41/50, waiting for 43 jobs, finished 2 jobs           
2025-07-23 15:49:23: Sobol, failed: 37, best VAL_ACC: 21.99, running 41 = ∑41/50, waiting for 41 jobs                            
2025-07-23 15:49:39: Sobol, failed: 37, best VAL_ACC: 21.99, running 41 = ∑41/50, new result: 10.09                              
2025-07-23 15:50:08: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, waiting for 41 jobs, finished 1 job            
2025-07-23 15:50:18: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, waiting for 40 jobs                            
2025-07-23 15:50:34: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, waiting for 40 jobs                            
2025-07-23 15:50:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, waiting for 40 jobs                            
2025-07-23 15:51:06: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, waiting for 40 jobs                            
2025-07-23 15:51:20: Sobol, failed: 37, best VAL_ACC: 21.99, running 40 = ∑40/50, new result: 10.28                              
2025-07-23 15:51:48: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 40 jobs, finished 1 job            
2025-07-23 15:51:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 15:52:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 15:52:27: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 15:52:42: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 15:52:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 39 = ∑39/50, new result: 9.74                               
2025-07-23 15:53:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job            
2025-07-23 15:53:55: Sobol, failed: 37, best VAL_ACC: 21.99, running 38 = ∑38/50, waiting for 38 jobs                            
2025-07-23 15:54:09: Sobol, failed: 37, best VAL_ACC: 21.99, running 38 = ∑38/50, new result: 0.11                               
2025-07-23 15:54:38: Sobol, failed: 37, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job            
2025-07-23 15:54:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 15:55:01: Sobol, failed: 37, best VAL_ACC: 21.99, running 37 = ∑37/50, new result: 10.28                              
2025-07-23 15:55:32: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job            
2025-07-23 15:55:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:55:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:56:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:56:29: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:56:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:57:00: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:57:15: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:57:30: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:57:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:58:00: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:58:16: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:58:33: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:58:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:59:05: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:59:20: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:59:35: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 15:59:50: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:00:05: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:00:22: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:00:37: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:00:52: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:01:11: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:01:27: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:01:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:01:57: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:02:13: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:02:28: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:02:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:03:00: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:03:15: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:03:30: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:03:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:04:01: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:04:16: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:04:32: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:04:53: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:05:09: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:05:25: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:05:40: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:05:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:06:11: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:06:26: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:06:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:06:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:07:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:07:27: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:07:43: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:07:58: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:08:13: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:08:32: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:08:49: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:09:04: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:09:20: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:09:37: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:09:52: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:10:07: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:10:23: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:10:38: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:10:53: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:11:09: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:11:29: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:11:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:12:02: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:12:17: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:12:32: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:12:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:13:03: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:13:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:13:37: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:13:56: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:14:11: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:14:26: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:14:41: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:14:57: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:15:12: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:15:28: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:15:43: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:15:58: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:16:13: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:16:28: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:16:43: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:16:58: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:17:13: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:17:30: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:17:45: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:18:00: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:18:15: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:18:31: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:18:47: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:19:03: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:19:19: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 16:19:34: Sobol, failed: 37, best VAL_ACC: 21.99, running 36 = ∑36/50, new result: 10.71                              
2025-07-23 16:20:02: Sobol, failed: 37, best VAL_ACC: 21.99, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job            
2025-07-23 16:20:11: Sobol, failed: 37, best VAL_ACC: 21.99, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:20:42: Sobol, failed: 37, best VAL_ACC: 21.99, timeout/running 10/25 = ∑35/50, job_failed                          
2025-07-23 16:21:55: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 35 jobs, finished 10 jobs          
2025-07-23 16:22:04: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:22:19: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:22:35: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:22:50: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:23:05: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:23:21: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:23:36: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:23:51: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:24:06: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:24:21: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:24:36: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:24:51: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:25:06: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:25:22: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:25:37: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:25:53: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:26:09: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:26:24: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:26:39: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:26:54: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:27:10: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:27:25: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:27:40: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:27:55: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:28:10: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:28:26: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:28:42: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:28:58: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:29:13: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:29:28: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:29:43: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:29:59: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:30:14: Sobol, failed: 47, best VAL_ACC: 21.99, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:47: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:48: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:49: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:50: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:30:51: Sobol, failed: 47, best VAL_ACC: 21.99, timeout 25 = ∑25/50, job_failed                                     
2025-07-23 16:33:22: Sobol, failed: 72, best VAL_ACC: 21.99, waiting for 25 jobs, finished 25 jobs                               
2025-07-23 16:38:20: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #1/50                                 
2025-07-23 16:38:35: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #2/50                                 
2025-07-23 16:38:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #3/50                                 
2025-07-23 16:39:05: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #4/50                                 
2025-07-23 16:39:19: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #5/50                                 
2025-07-23 16:39:33: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #6/50                                 
2025-07-23 16:39:47: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #7/50                                 
2025-07-23 16:40:02: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #8/50                                 
2025-07-23 16:40:16: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #9/50                                 
2025-07-23 16:40:29: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #10/50                                
2025-07-23 16:41:01: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #11/50                                
2025-07-23 16:41:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #12/50                                
2025-07-23 16:41:28: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #13/50                                
2025-07-23 16:41:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #14/50                                
2025-07-23 16:41:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #15/50                                
2025-07-23 16:42:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #16/50                                
2025-07-23 16:42:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #17/50                                
2025-07-23 16:42:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #18/50                                
2025-07-23 16:42:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #19/50                                
2025-07-23 16:43:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #20/50                                
2025-07-23 16:43:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #21/50                                
2025-07-23 16:43:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #22/50                                
2025-07-23 16:44:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #23/50                                
2025-07-23 16:44:40: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #24/50                                
2025-07-23 16:44:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #25/50                                
2025-07-23 16:45:09: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #26/50                                
2025-07-23 16:45:26: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #27/50                                
2025-07-23 16:45:42: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #28/50                                
2025-07-23 16:45:59: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #29/50                                
2025-07-23 16:46:15: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #30/50                                
2025-07-23 16:46:32: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #31/50                                
2025-07-23 16:46:47: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #32/50                                
2025-07-23 16:47:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #33/50                                
2025-07-23 16:47:18: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #34/50                                
2025-07-23 16:47:32: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #35/50                                
2025-07-23 16:47:49: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #36/50                                
2025-07-23 16:48:05: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #37/50                                
2025-07-23 16:48:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #38/50                                
2025-07-23 16:50:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #39/50                                
2025-07-23 16:50:19: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #40/50                                
2025-07-23 16:50:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #41/50                                
2025-07-23 16:50:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #42/50                                
2025-07-23 16:51:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #43/50                                
2025-07-23 16:51:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #44/50                                
2025-07-23 16:51:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #45/50                                
2025-07-23 16:51:56: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #46/50                                
2025-07-23 16:52:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #47/50                                
2025-07-23 16:52:28: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #48/50                                
2025-07-23 16:52:44: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #49/50                                
2025-07-23 16:52:59: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, getting new HP set #50/50                                
2025-07-23 16:53:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, requested 50 jobs, got 50, 23.64 s/job                   
2025-07-23 16:53:21: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #1/50 start                                         
2025-07-23 16:53:30: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #2/50 start                                         
2025-07-23 16:53:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #3/50 start                                         
2025-07-23 16:53:47: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #4/50 start                                         
2025-07-23 16:53:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #5/50 start                                         
2025-07-23 16:54:02: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #6/50 start                                         
2025-07-23 16:54:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #7/50 start                                         
2025-07-23 16:54:18: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #8/50 start                                         
2025-07-23 16:54:27: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #9/50 start                                         
2025-07-23 16:54:35: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #10/50 start                                        
2025-07-23 16:54:44: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #11/50 start                                        
2025-07-23 16:54:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #12/50 start                                        
2025-07-23 16:55:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #13/50 start                                        
2025-07-23 16:55:13: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #14/50 start                                        
2025-07-23 16:55:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #15/50 start                                        
2025-07-23 16:55:32: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #16/50 start                                        
2025-07-23 16:55:40: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #17/50 start                                        
2025-07-23 16:57:17: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #18/50 start                                        
2025-07-23 16:57:25: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #19/50 start                                        
2025-07-23 16:57:34: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #20/50 start                                        
2025-07-23 16:57:42: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #21/50 start                                        
2025-07-23 16:57:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #22/50 start                                        
2025-07-23 16:57:58: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #23/50 start                                        
2025-07-23 16:58:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #24/50 start                                        
2025-07-23 16:58:13: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #25/50 start                                        
2025-07-23 16:58:21: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #26/50 start                                        
2025-07-23 16:58:29: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #27/50 start                                        
2025-07-23 16:58:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #28/50 start                                        
2025-07-23 16:58:43: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #29/50 start                                        
2025-07-23 16:58:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #30/50 start                                        
2025-07-23 16:58:58: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #31/50 start                                        
2025-07-23 16:59:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #32/50 start                                        
2025-07-23 16:59:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #33/50 start                                        
2025-07-23 16:59:22: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #34/50 start                                        
2025-07-23 16:59:30: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #35/50 start                                        
2025-07-23 16:59:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #36/50 start                                        
2025-07-23 16:59:44: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #37/50 start                                        
2025-07-23 16:59:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #38/50 start                                        
2025-07-23 16:59:58: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #39/50 start                                        
2025-07-23 17:00:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #40/50 start                                        
2025-07-23 17:00:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #41/50 start                                        
2025-07-23 17:00:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #42/50 start                                        
2025-07-23 17:00:31: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #43/50 start                                        
2025-07-23 17:00:38: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #44/50 start                                        
2025-07-23 17:00:45: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #45/50 start                                        
2025-07-23 17:00:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #46/50 start                                        
2025-07-23 17:00:59: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #47/50 start                                        
2025-07-23 17:01:08: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #48/50 start                                        
2025-07-23 17:01:17: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #49/50 start                                        
2025-07-23 17:01:27: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, eval #50/50 start                                        
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:01:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, starting new job                                         
2025-07-23 17:02:47: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, unknown 5 = ∑5/50, started new job                       
2025-07-23 17:02:48: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, unknown 7 = ∑7/50, started new job                       
2025-07-23 17:02:49: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, unknown 7 = ∑7/50, started new job                       
2025-07-23 17:02:49: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/2 = ∑8/50, started new job             
2025-07-23 17:02:49: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/3 = ∑9/50, started new job             
2025-07-23 17:02:49: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/4 = ∑10/50, started new job            
2025-07-23 17:02:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/5 = ∑11/50, started new job            
2025-07-23 17:02:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/9 = ∑15/50, started new job            
2025-07-23 17:02:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/9 = ∑15/50, started new job            
2025-07-23 17:02:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:02:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, pending/unknown 6/10 = ∑16/50, started new job           
2025-07-23 17:03:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job                    
2025-07-23 17:04:00: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job                    
2025-07-23 17:04:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 16 = ∑16/50, starting new job                    
2025-07-23 17:04:16: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 16/1 = ∑17/50, started new job           
2025-07-23 17:04:17: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 16/1 = ∑17/50, starting new job          
2025-07-23 17:04:18: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:19: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, started new job 
2025-07-23 17:04:22: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:22: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:25: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:25: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 16/1/1 = ∑18/50, starting new job
2025-07-23 17:04:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 18/1 = ∑19/50, started new job           
2025-07-23 17:05:25: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 19/2 = ∑21/50, started new job           
2025-07-23 17:05:27: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 19/2 = ∑21/50, started new job           
2025-07-23 17:05:28: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 19/2 = ∑21/50, starting new job          
2025-07-23 17:05:28: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 19/2 = ∑21/50, starting new job          
2025-07-23 17:05:30: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 19/2/1 = ∑22/50, started new job 
2025-07-23 17:05:34: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 19/3/1 = ∑23/50, starting new job
2025-07-23 17:05:35: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 19/3/1 = ∑23/50, started new job 
2025-07-23 17:05:35: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 19/3/1 = ∑23/50, starting new job
2025-07-23 17:05:40: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 22/8 = ∑30/50, started new job           
2025-07-23 17:05:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 22/8 = ∑30/50, started new job           
2025-07-23 17:05:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 22/8 = ∑30/50, started new job           
2025-07-23 17:05:40: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 22/8 = ∑30/50, started new job           
2025-07-23 17:05:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 23/7 = ∑30/50, started new job           
2025-07-23 17:05:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 23/7 = ∑30/50, started new job           
2025-07-23 17:05:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 23/7 = ∑30/50, started new job           
2025-07-23 17:05:44: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 23/7 = ∑30/50, starting new job          
2025-07-23 17:06:26: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 30/2 = ∑32/50, started new job           
2025-07-23 17:06:26: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 30/2 = ∑32/50, starting new job          
2025-07-23 17:06:26: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 30/2 = ∑32/50, started new job           
2025-07-23 17:06:29: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 30/2 = ∑32/50, starting new job          
2025-07-23 17:06:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 30/2 = ∑32/50, starting new job          
2025-07-23 17:06:47: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 32/2 = ∑34/50, started new job           
2025-07-23 17:06:48: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 32/2 = ∑34/50, started new job           
2025-07-23 17:06:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/2 = ∑34/50, starting new job          
2025-07-23 17:06:58: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, started new job           
2025-07-23 17:07:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 32/3 = ∑35/50, starting new job          
2025-07-23 17:07:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 35/2 = ∑37/50, started new job           
2025-07-23 17:07:43: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 35/2 = ∑37/50, started new job           
2025-07-23 17:07:44: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 35/2 = ∑37/50, starting new job          
2025-07-23 17:07:45: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 35/2 = ∑37/50, starting new job          
2025-07-23 17:07:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 37/1 = ∑38/50, started new job           
2025-07-23 17:08:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 37/1/1 = ∑39/50, started new job 
2025-07-23 17:08:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 37/2 = ∑39/50, starting new job          
2025-07-23 17:08:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending 37/2 = ∑39/50, starting new job          
2025-07-23 17:08:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 39/7 = ∑46/50, started new job           
2025-07-23 17:08:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 46/2 = ∑48/50, started new job           
2025-07-23 17:08:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/unknown 46/2 = ∑48/50, started new job           
2025-07-23 17:09:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 46/2/2 = ∑50/50, started new job 
2025-07-23 17:09:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running/pending/unknown 46/2/2 = ∑50/50, started new job 
2025-07-23 17:09:48: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:10:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:10:17: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:10:32: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:10:46: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:11:00: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:11:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:11:28: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:11:43: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:11:57: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:12:12: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:12:26: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:12:41: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:12:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:13:10: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:13:25: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:13:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:13:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:14:08: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:14:22: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:14:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:14:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:15:07: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:15:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:15:38: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:15:52: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:16:07: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:16:21: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:16:36: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:16:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:17:04: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:17:20: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:17:37: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:17:51: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:18:06: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:18:21: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:18:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:18:55: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:19:09: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:19:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:19:38: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:19:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:20:09: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:20:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:20:38: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:20:53: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:21:08: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:21:23: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:21:39: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:21:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:22:09: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:22:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:22:38: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:22:53: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:23:09: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:23:24: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:23:40: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, waiting for 50 jobs                 
2025-07-23 17:23:54: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 50 = ∑50/50, new result: 0.19                    
2025-07-23 17:24:22: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job 
2025-07-23 17:24:30: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:24:45: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:25:01: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:25:15: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:25:29: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:25:43: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:25:59: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, waiting for 49 jobs                 
2025-07-23 17:26:14: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 49 = ∑49/50, new result: 6.65                    
2025-07-23 17:26:42: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job 
2025-07-23 17:26:50: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 48 = ∑48/50, waiting for 48 jobs                 
2025-07-23 17:27:03: BoTorchGenerator, failed: 72, best VAL_ACC: 21.99, running 48 = ∑48/50, new result: 39.15                   
2025-07-23 17:27:30: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job 
2025-07-23 17:27:38: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:27:53: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:28:07: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:28:22: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:28:37: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:28:52: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:29:07: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:29:21: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:29:35: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:29:50: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, waiting for 47 jobs                 
2025-07-23 17:30:04: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 47 = ∑47/50, new result: 0.29                    
2025-07-23 17:30:40: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running/completed 44/2 = ∑46/50, waiting for 47 jobs, finished 1 job
2025-07-23 17:30:49: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running/completed 44/2 = ∑46/50, waiting for 46 jobs                
2025-07-23 17:31:04: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running/completed 44/2 = ∑46/50, new result: 2.65                   
2025-07-23 17:31:04: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running/completed 44/2 = ∑46/50, new result: 26.94                  
2025-07-23 17:31:04: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running/completed 44/2 = ∑46/50, new result: 17.72                  
2025-07-23 17:32:02: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 43 = ∑43/50, waiting for 46 jobs, finished 3 jobs           
2025-07-23 17:32:12: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 43 = ∑43/50, waiting for 43 jobs                            
2025-07-23 17:32:25: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 43 = ∑43/50, new result: 18.81                              
2025-07-23 17:32:26: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 43 = ∑43/50, new result: 11.35                              
2025-07-23 17:33:14: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 41 = ∑41/50, waiting for 43 jobs, finished 2 jobs           
2025-07-23 17:33:22: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 41 = ∑41/50, waiting for 41 jobs                            
2025-07-23 17:33:36: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 41 = ∑41/50, new result: 79.54                              
2025-07-23 17:33:36: BoTorchGenerator, failed: 72, best VAL_ACC: 39.15, running 41 = ∑41/50, new result: 30.54                              
2025-07-23 17:34:15: BoTorchGenerator, failed: 72, best VAL_ACC: 79.54, running 39 = ∑39/50, waiting for 41 jobs, finished 2 jobs           
2025-07-23 17:34:24: BoTorchGenerator, failed: 72, best VAL_ACC: 79.54, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 17:34:42: BoTorchGenerator, failed: 72, best VAL_ACC: 79.54, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 17:35:00: BoTorchGenerator, failed: 72, best VAL_ACC: 79.54, running 39 = ∑39/50, waiting for 39 jobs                            
2025-07-23 17:35:17: BoTorchGenerator, failed: 72, best VAL_ACC: 79.54, running 39 = ∑39/50, new result: 98.16                              
2025-07-23 17:35:51: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 38 = ∑38/50, waiting for 39 jobs, finished 1 job            
2025-07-23 17:36:00: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 38 = ∑38/50, waiting for 38 jobs                            
2025-07-23 17:36:14: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 38 = ∑38/50, new result: 41.32                              
2025-07-23 17:36:43: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job            
2025-07-23 17:36:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:37:07: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:37:22: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:37:38: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:37:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:38:10: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:38:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:38:43: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 17:38:58: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 37 = ∑37/50, new result: 97.91                              
2025-07-23 17:39:27: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job            
2025-07-23 17:39:35: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 17:39:50: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 17:40:04: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 17:40:20: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 17:40:36: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 36 = ∑36/50, new result: 0.08                               
2025-07-23 17:41:09: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 36 jobs, finished 1 job            
2025-07-23 17:41:20: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:41:36: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:41:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:42:09: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:42:24: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:42:40: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:42:58: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:43:13: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:43:28: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:43:45: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:44:03: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:44:20: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:44:40: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:44:58: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:45:13: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:45:29: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, waiting for 35 jobs                            
2025-07-23 17:45:45: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 35 = ∑35/50, new result: 0.2                                
2025-07-23 17:46:14: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 35 jobs, finished 1 job            
2025-07-23 17:46:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 17:46:41: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 17:46:56: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 34 = ∑34/50, new result: 9.8                                
2025-07-23 17:47:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job            
2025-07-23 17:47:34: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:47:49: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:48:04: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:48:21: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:48:36: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:48:51: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:49:09: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:49:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:49:40: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:49:56: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:50:12: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:50:28: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:50:44: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:51:01: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:51:17: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:51:33: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:51:50: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 17:52:05: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 33 = ∑33/50, new result: 0.35                               
2025-07-23 17:52:32: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job            
2025-07-23 17:52:41: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:52:55: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:53:11: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:53:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:53:41: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:53:56: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:54:11: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:54:26: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:54:41: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:54:56: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:55:11: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:55:26: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:55:42: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:55:57: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:56:13: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:56:28: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:56:43: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:56:59: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:57:13: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:57:28: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:57:45: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:58:00: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:58:16: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:58:31: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:58:49: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 17:59:02: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 32 = ∑32/50, new result: 13.82                              
2025-07-23 17:59:31: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job            
2025-07-23 17:59:40: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 17:59:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:00:09: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:00:23: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:00:38: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:00:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:01:08: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:01:23: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:01:39: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:01:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:02:10: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:02:25: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:02:39: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:02:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:03:10: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:03:26: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:03:42: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:03:57: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:04:12: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:04:27: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 18:04:41: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 31 = ∑31/50, new result: 0.1                                
2025-07-23 18:05:08: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job            
2025-07-23 18:05:17: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:05:32: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:05:48: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:06:02: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:06:17: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:06:32: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:06:48: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:07:02: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:07:18: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:07:34: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:07:50: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:08:07: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:08:22: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:08:37: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:08:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:09:08: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:09:23: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:09:40: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:09:56: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:10:19: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:10:35: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:10:50: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:11:05: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:11:20: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:53: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:52: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:54: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:55: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:55: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:11:58: BoTorchGenerator, failed: 72, best VAL_ACC: 98.16, timeout 30 = ∑30/50, job_failed                                     
2025-07-23 18:15:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, waiting for 30 jobs, finished 30 jobs                              
2025-07-23 18:29:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #1/50                                           
2025-07-23 18:30:15: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #2/50                                           
2025-07-23 18:30:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #3/50                                           
2025-07-23 18:31:02: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #4/50                                           
2025-07-23 18:31:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #5/50                                           
2025-07-23 18:31:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #6/50                                           
2025-07-23 18:32:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #7/50                                           
2025-07-23 18:32:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #8/50                                           
2025-07-23 18:33:00: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #9/50                                           
2025-07-23 18:33:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #10/50                                          
2025-07-23 18:33:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #11/50                                          
2025-07-23 18:34:11: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #12/50                                          
2025-07-23 18:34:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #13/50                                          
2025-07-23 18:34:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #14/50                                          
2025-07-23 18:35:21: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #15/50                                          
2025-07-23 18:35:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #16/50                                          
2025-07-23 18:36:07: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #17/50                                          
2025-07-23 18:36:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #18/50                                          
2025-07-23 18:36:57: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #19/50                                          
2025-07-23 18:37:20: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #20/50                                          
2025-07-23 18:37:45: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #21/50                                          
2025-07-23 18:38:13: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #22/50                                          
2025-07-23 18:38:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #23/50                                          
2025-07-23 18:39:05: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #24/50                                          
2025-07-23 18:39:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #25/50                                          
2025-07-23 18:39:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #26/50                                          
2025-07-23 18:40:25: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #27/50                                          
2025-07-23 18:40:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #28/50                                          
2025-07-23 18:41:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #29/50                                          
2025-07-23 18:41:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #30/50                                          
2025-07-23 18:41:59: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #31/50                                          
2025-07-23 18:42:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #32/50                                          
2025-07-23 18:42:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #33/50                                          
2025-07-23 18:43:09: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #34/50                                          
2025-07-23 18:43:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #35/50                                          
2025-07-23 18:43:57: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #36/50                                          
2025-07-23 18:44:20: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #37/50                                          
2025-07-23 18:44:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #38/50                                          
2025-07-23 18:45:06: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #39/50                                          
2025-07-23 18:45:29: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #40/50                                          
2025-07-23 18:45:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #41/50                                          
2025-07-23 18:46:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #42/50                                          
2025-07-23 18:46:39: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #43/50                                          
2025-07-23 18:47:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #44/50                                          
2025-07-23 18:47:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #45/50                                          
2025-07-23 18:47:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #46/50                                          
2025-07-23 18:48:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #47/50                                          
2025-07-23 18:48:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #48/50                                          
2025-07-23 18:49:00: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #49/50                                          
2025-07-23 18:49:24: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, getting new HP set #50/50                                          
2025-07-23 18:49:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, requested 50 jobs, got 50, 40.84 s/job                             
2025-07-23 18:49:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #1/50 start                                                   
2025-07-23 18:50:10: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #2/50 start                                                   
2025-07-23 18:50:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #3/50 start                                                   
2025-07-23 18:50:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #4/50 start                                                   
2025-07-23 18:50:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #5/50 start                                                   
2025-07-23 18:51:02: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #6/50 start                                                   
2025-07-23 18:51:14: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #7/50 start                                                   
2025-07-23 18:51:27: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #8/50 start                                                   
2025-07-23 18:51:39: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #9/50 start                                                   
2025-07-23 18:51:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #10/50 start                                                  
2025-07-23 18:52:05: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #11/50 start                                                  
2025-07-23 18:52:18: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #12/50 start                                                  
2025-07-23 18:52:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #13/50 start                                                  
2025-07-23 18:52:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #14/50 start                                                  
2025-07-23 18:52:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #15/50 start                                                  
2025-07-23 18:53:08: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #16/50 start                                                  
2025-07-23 18:53:21: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #17/50 start                                                  
2025-07-23 18:53:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #18/50 start                                                  
2025-07-23 18:53:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #19/50 start                                                  
2025-07-23 18:54:00: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #20/50 start                                                  
2025-07-23 18:54:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #21/50 start                                                  
2025-07-23 18:54:25: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #22/50 start                                                  
2025-07-23 18:54:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #23/50 start                                                  
2025-07-23 18:54:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #24/50 start                                                  
2025-07-23 18:55:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #25/50 start                                                  
2025-07-23 18:55:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #26/50 start                                                  
2025-07-23 18:55:29: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #27/50 start                                                  
2025-07-23 18:55:42: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #28/50 start                                                  
2025-07-23 18:55:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #29/50 start                                                  
2025-07-23 18:56:07: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #30/50 start                                                  
2025-07-23 18:56:20: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #31/50 start                                                  
2025-07-23 18:56:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #32/50 start                                                  
2025-07-23 18:56:45: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #33/50 start                                                  
2025-07-23 18:56:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #34/50 start                                                  
2025-07-23 18:57:10: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #35/50 start                                                  
2025-07-23 18:57:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #36/50 start                                                  
2025-07-23 18:57:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #37/50 start                                                  
2025-07-23 18:57:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #38/50 start                                                  
2025-07-23 18:58:01: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #39/50 start                                                  
2025-07-23 18:58:13: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #40/50 start                                                  
2025-07-23 18:58:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #41/50 start                                                  
2025-07-23 18:58:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #42/50 start                                                  
2025-07-23 18:58:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #43/50 start                                                  
2025-07-23 18:59:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #44/50 start                                                  
2025-07-23 18:59:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #45/50 start                                                  
2025-07-23 18:59:29: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #46/50 start                                                  
2025-07-23 18:59:42: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #47/50 start                                                  
2025-07-23 18:59:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #48/50 start                                                  
2025-07-23 19:00:08: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #49/50 start                                                  
2025-07-23 19:00:21: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, eval #50/50 start                                                  
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:00:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, starting new job                                                   
2025-07-23 19:02:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, unknown 3 = ∑3/50, started new job                                 
2025-07-23 19:02:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, unknown 3 = ∑3/50, started new job                                 
2025-07-23 19:02:41: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending/unknown 3/2 = ∑5/50, started new job                       
2025-07-23 19:02:42: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending/unknown 3/2 = ∑5/50, started new job                       
2025-07-23 19:02:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending/unknown 3/10 = ∑13/50, started new job                     
2025-07-23 19:02:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending/unknown 3/12 = ∑15/50, started new job                     
2025-07-23 19:02:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending/unknown 3/12 = ∑15/50, started new job                     
2025-07-23 19:02:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 15 = ∑15/50, started new job                               
2025-07-23 19:02:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 15 = ∑15/50, started new job                               
2025-07-23 19:02:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 15 = ∑15/50, started new job                               
2025-07-23 19:02:46: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 15 = ∑15/50, started new job                               
2025-07-23 19:02:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 16 = ∑16/50, started new job                               
2025-07-23 19:02:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 16 = ∑16/50, started new job                               
2025-07-23 19:02:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 16 = ∑16/50, started new job                               
2025-07-23 19:02:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 16 = ∑16/50, started new job                               
2025-07-23 19:02:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, pending 16 = ∑16/50, started new job                               
2025-07-23 19:05:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:22: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:25: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:27: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:29: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:05:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:06:22: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:06:41: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 15/1 = ∑16/50, starting new job                  
2025-07-23 19:06:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 15/1/2 = ∑18/50, started new job         
2025-07-23 19:06:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 15/1/2 = ∑18/50, started new job         
2025-07-23 19:07:28: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 17/1/1 = ∑19/50, started new job         
2025-07-23 19:07:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 17/1/1/3 = ∑22/50, started new job
2025-07-23 19:07:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 17/1/1/3 = ∑22/50, started new job
2025-07-23 19:07:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending 17/1/4 = ∑22/50, started new job          
2025-07-23 19:07:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 17/1/4/1 = ∑23/50, started new job
2025-07-23 19:07:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 23/1 = ∑24/50, started new job                    
2025-07-23 19:07:42: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 23/1/6 = ∑30/50, started new job          
2025-07-23 19:07:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 23/1/5 = ∑29/50, started new job          
2025-07-23 19:07:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 23/1/6 = ∑30/50, started new job          
2025-07-23 19:07:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 23/1/6 = ∑30/50, started new job          
2025-07-23 19:07:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 23/1/6 = ∑30/50, started new job          
2025-07-23 19:07:45: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending 23/1/6 = ∑30/50, started new job          
2025-07-23 19:08:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 29/1/1 = ∑31/50, started new job          
2025-07-23 19:08:45: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 29/2/1 = ∑32/50, started new job          
2025-07-23 19:09:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 30/2 = ∑32/50, starting new job                   
2025-07-23 19:10:06: BoTorchGenerator, failed: 102, best , running/completed 28/4 = ∑32/50, starting new job                                 
2025-07-23 19:10:06: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:18: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:32: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:32: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:32: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:10:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:11:06: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:11:24: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 28/4 = ∑32/50, starting new job                   
2025-07-23 19:11:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 28/4/1 = ∑33/50, started new job          
2025-07-23 19:12:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 26/7 = ∑33/50, starting new job                   
2025-07-23 19:12:25: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 26/7/2 = ∑35/50, started new job          
2025-07-23 19:12:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 26/7/2 = ∑35/50, started new job          
2025-07-23 19:12:39: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 26/7/2/2 = ∑37/50, started new job
2025-07-23 19:12:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 26/7/2/2 = ∑37/50, started new job
2025-07-23 19:12:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 26/7/4/1 = ∑38/50, started new job
2025-07-23 19:12:54: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 26/7/4/2 = ∑39/50, started new job
2025-07-23 19:12:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 30/7/5 = ∑42/50, started new job          
2025-07-23 19:12:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 30/7/6 = ∑43/50, started new job          
2025-07-23 19:12:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 35/7/1 = ∑43/50, started new job          
2025-07-23 19:12:57: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 35/7/3 = ∑45/50, started new job          
2025-07-23 19:12:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 35/7/3 = ∑45/50, started new job          
2025-07-23 19:12:59: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 35/7/3 = ∑45/50, started new job          
2025-07-23 19:13:27: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 38/7/1 = ∑46/50, started new job          
2025-07-23 19:13:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending/unknown 38/7/1/1 = ∑47/50, started new job
2025-07-23 19:14:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 39/8/1 = ∑48/50, started new job          
2025-07-23 19:14:54: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending 39/8/1 = ∑48/50, starting new job         
2025-07-23 19:14:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/pending 39/8/1 = ∑48/50, starting new job         
2025-07-23 19:16:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 37/11/2 = ∑50/50, started new job         
2025-07-23 19:16:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed/unknown 37/11/2 = ∑50/50, started new job         
2025-07-23 19:17:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 83.58                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 95.36                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 96.58                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 94.59                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 36.14                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 73.07                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 92.09                 
2025-07-23 19:17:36: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 97.64                 
2025-07-23 19:17:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 29.74                 
2025-07-23 19:17:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 83.96                 
2025-07-23 19:17:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 39/11 = ∑50/50, new result: 35.87                 
2025-07-23 19:21:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 37/2 = ∑39/50, finishing jobs, finished 11 jobs   
2025-07-23 19:21:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 37/2 = ∑39/50, new result: 84.52                  
2025-07-23 19:21:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running/completed 37/2 = ∑39/50, new result: 9.74                   
2025-07-23 19:23:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 37 = ∑37/50, finishing previous jobs (39), finished 2 jobs  
2025-07-23 19:23:18: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 37 = ∑37/50, waiting for 37 jobs                            
2025-07-23 19:23:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 37 = ∑37/50, new result: 97.91                              
2025-07-23 19:24:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 37 jobs, finished 1 job            
2025-07-23 19:24:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 36 = ∑36/50, waiting for 36 jobs                            
2025-07-23 19:25:07: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 36 = ∑36/50, new result: 84.31                              
2025-07-23 19:25:07: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 36 = ∑36/50, new result: 80.04                              
2025-07-23 19:26:15: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 36 jobs, finished 2 jobs           
2025-07-23 19:26:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 19:26:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 19:27:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 19:27:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, waiting for 34 jobs                            
2025-07-23 19:27:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 34 = ∑34/50, new result: 97.64                              
2025-07-23 19:28:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 34 jobs, finished 1 job            
2025-07-23 19:28:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:29:11: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:29:32: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:29:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:30:13: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:30:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:30:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, waiting for 33 jobs                            
2025-07-23 19:31:14: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 33 = ∑33/50, new result: 77.34                              
2025-07-23 19:31:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 33 jobs, finished 1 job            
2025-07-23 19:32:11: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 32 = ∑32/50, waiting for 32 jobs                            
2025-07-23 19:32:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 32 = ∑32/50, new result: 86.6                               
2025-07-23 19:33:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 32 jobs, finished 1 job            
2025-07-23 19:33:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:33:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:34:12: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:34:33: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:34:54: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:35:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, waiting for 31 jobs                            
2025-07-23 19:35:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 31 = ∑31/50, new result: 97.63                              
2025-07-23 19:36:20: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job            
2025-07-23 19:36:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:36:55: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:37:16: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:37:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:37:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:38:19: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:38:41: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:39:02: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:39:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:39:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:40:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:40:26: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:40:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:41:07: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:41:28: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:41:49: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:42:09: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:42:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:42:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:43:13: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:43:35: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:43:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:44:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:44:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:44:59: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, waiting for 30 jobs                            
2025-07-23 19:45:18: BoTorchGenerator, failed: 102, best VAL_ACC: 98.16, running 30 = ∑30/50, new result: 98.4                               
2025-07-23 19:46:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 30 jobs, finished 1 job             
2025-07-23 19:46:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:46:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:46:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:47:19: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:47:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:48:02: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:48:23: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:48:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, waiting for 29 jobs                             
2025-07-23 19:49:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 29 = ∑29/50, new result: 86.58                               
2025-07-23 19:49:47: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 28 = ∑28/50, waiting for 29 jobs, finished 1 job             
2025-07-23 19:50:01: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 28 = ∑28/50, waiting for 28 jobs                             
2025-07-23 19:50:22: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 28 = ∑28/50, waiting for 28 jobs                             
2025-07-23 19:50:40: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 28 = ∑28/50, new result: 98.2                                
2025-07-23 19:51:24: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 27 = ∑27/50, waiting for 28 jobs, finished 1 job             
2025-07-23 19:51:39: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 27 = ∑27/50, waiting for 27 jobs                             
2025-07-23 19:51:58: BoTorchGenerator, failed: 102, best VAL_ACC: 98.4, running 27 = ∑27/50, new result: 98.44                               
2025-07-23 19:52:44: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 26 = ∑26/50, waiting for 27 jobs, finished 1 job            
2025-07-23 19:52:57: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 26 = ∑26/50, waiting for 26 jobs                            
2025-07-23 19:53:18: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 26 = ∑26/50, waiting for 26 jobs                            
2025-07-23 19:53:39: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 26 = ∑26/50, new result: 90.04                              
2025-07-23 19:54:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 25 = ∑25/50, waiting for 26 jobs, finished 1 job            
2025-07-23 19:54:51: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 25 = ∑25/50, waiting for 25 jobs                            
2025-07-23 19:55:10: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 25 = ∑25/50, new result: 96.83                              
2025-07-23 19:55:10: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 25 = ∑25/50, new result: 98.16                              
2025-07-23 19:55:10: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 25 = ∑25/50, new result: 84.34                              
2025-07-23 19:57:01: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running/completed 21/1 = ∑22/50, waiting for 25 jobs, finished 3 jobs
2025-07-23 19:57:15: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running/completed 21/1 = ∑22/50, waiting for 22 jobs                 
2025-07-23 19:57:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running/completed 21/1 = ∑22/50, new result: 98.29                   
2025-07-23 19:57:34: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running/completed 21/1 = ∑22/50, new result: 98.2                    
2025-07-23 19:58:42: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 22 jobs, finished 2 jobs            
2025-07-23 19:58:56: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 19:59:17: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 19:59:38: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:00:00: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:00:22: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:00:43: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:01:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:01:25: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:01:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:02:09: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:02:31: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:02:52: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:03:14: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:03:37: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:03:59: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:04:20: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:04:41: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:05:03: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:05:24: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:05:48: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:06:09: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:06:30: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, running 20 = ∑20/50, waiting for 20 jobs                             
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:04: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:07:05: BoTorchGenerator, failed: 102, best VAL_ACC: 98.44, timeout/running 7/13 = ∑20/50, job_failed                            
2025-07-23 20:08:30: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 20 jobs, finished 7 jobs            
2025-07-23 20:08:44: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:09:07: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:09:28: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:09:49: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:10:10: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:10:32: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:10:54: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:11:17: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:11:38: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:12:00: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:12:23: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:12:44: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:13:05: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:13:26: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:13:48: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:14:09: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:14:31: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:14:52: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:15:14: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:15:36: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:15:57: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:16:18: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:16:40: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, running 13 = ∑13/50, waiting for 13 jobs                             
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:15: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:16: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:16: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:16: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:17:16: BoTorchGenerator, failed: 109, best VAL_ACC: 98.44, timeout/running 12/1 = ∑13/50, job_failed                            
2025-07-23 20:19:31: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 13 jobs, finished 12 jobs             
2025-07-23 20:19:46: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:20:08: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:20:29: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:20:50: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:21:13: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:21:34: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:21:55: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:22:16: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:22:37: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:22:59: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:23:20: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:23:41: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:24:02: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:24:23: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:24:44: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:25:05: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:25:25: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:25:46: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:26:07: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:26:28: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:26:50: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, running 1 = ∑1/50, waiting for 1 job                                 
2025-07-23 20:27:24: BoTorchGenerator, failed: 121, best VAL_ACC: 98.44, timeout 1 = ∑1/50, job_failed                                        
2025-07-23 20:27:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, waiting for 1 job, finished 1 job                                    
2025-07-23 20:39:19: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #1/50                                             
2025-07-23 20:39:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #2/50                                             
2025-07-23 20:40:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #3/50                                             
2025-07-23 20:40:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #4/50                                             
2025-07-23 20:41:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #5/50                                             
2025-07-23 20:41:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #6/50                                             
2025-07-23 20:42:10: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #7/50                                             
2025-07-23 20:42:41: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #8/50                                             
2025-07-23 20:43:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #9/50                                             
2025-07-23 20:43:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #10/50                                            
2025-07-23 20:44:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #11/50                                            
2025-07-23 20:44:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #12/50                                            
2025-07-23 20:45:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #12/50                                            
2025-07-23 20:45:34: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #13/50                                            
2025-07-23 20:46:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #14/50                                            
2025-07-23 20:46:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #15/50                                            
2025-07-23 20:47:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #16/50                                            
2025-07-23 20:47:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #17/50                                            
2025-07-23 20:48:08: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #18/50                                            
2025-07-23 20:48:40: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #19/50                                            
2025-07-23 20:49:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #20/50                                            
2025-07-23 20:49:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #21/50                                            
2025-07-23 20:50:16: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #22/50                                            
2025-07-23 20:50:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #23/50                                            
2025-07-23 20:51:22: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #24/50                                            
2025-07-23 20:51:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #25/50                                            
2025-07-23 20:54:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #26/50                                            
2025-07-23 20:55:16: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #27/50                                            
2025-07-23 20:55:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #28/50                                            
2025-07-23 20:56:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #29/50                                            
2025-07-23 20:56:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #30/50                                            
2025-07-23 20:57:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #31/50                                            
2025-07-23 20:57:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #32/50                                            
2025-07-23 20:58:07: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #33/50                                            
2025-07-23 20:58:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #34/50                                            
2025-07-23 20:59:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #35/50                                            
2025-07-23 20:59:34: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #36/50                                            
2025-07-23 21:00:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #37/50                                            
2025-07-23 21:00:34: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #38/50                                            
2025-07-23 21:01:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #39/50                                            
2025-07-23 21:01:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #40/50                                            
2025-07-23 21:02:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #41/50                                            
2025-07-23 21:02:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #42/50                                            
2025-07-23 21:03:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #43/50                                            
2025-07-23 21:03:30: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #44/50                                            
2025-07-23 21:04:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #45/50                                            
2025-07-23 21:04:30: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #46/50                                            
2025-07-23 21:05:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #47/50                                            
2025-07-23 21:05:29: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #48/50                                            
2025-07-23 21:05:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #49/50                                            
2025-07-23 21:07:09: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #49/50                                            
2025-07-23 21:07:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, getting new HP set #50/50                                            
2025-07-23 21:08:09: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, requested 50 jobs, got 50, 47.94 s/job                               
2025-07-23 21:08:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #1/50 start                                                     
2025-07-23 21:08:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #2/50 start                                                     
2025-07-23 21:08:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #3/50 start                                                     
2025-07-23 21:09:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #4/50 start                                                     
2025-07-23 21:09:30: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #5/50 start                                                     
2025-07-23 21:09:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #6/50 start                                                     
2025-07-23 21:10:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #7/50 start                                                     
2025-07-23 21:10:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #8/50 start                                                     
2025-07-23 21:10:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #9/50 start                                                     
2025-07-23 21:10:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #10/50 start                                                    
2025-07-23 21:11:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #11/50 start                                                    
2025-07-23 21:11:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #12/50 start                                                    
2025-07-23 21:11:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #13/50 start                                                    
2025-07-23 21:11:53: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #14/50 start                                                    
2025-07-23 21:12:10: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #15/50 start                                                    
2025-07-23 21:12:28: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #16/50 start                                                    
2025-07-23 21:12:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #17/50 start                                                    
2025-07-23 21:13:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #18/50 start                                                    
2025-07-23 21:13:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #19/50 start                                                    
2025-07-23 21:13:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #20/50 start                                                    
2025-07-23 21:13:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #21/50 start                                                    
2025-07-23 21:14:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #22/50 start                                                    
2025-07-23 21:14:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #23/50 start                                                    
2025-07-23 21:14:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #24/50 start                                                    
2025-07-23 21:15:08: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #25/50 start                                                    
2025-07-23 21:15:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #26/50 start                                                    
2025-07-23 21:15:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #27/50 start                                                    
2025-07-23 21:16:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #28/50 start                                                    
2025-07-23 21:16:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #29/50 start                                                    
2025-07-23 21:16:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #30/50 start                                                    
2025-07-23 21:16:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #31/50 start                                                    
2025-07-23 21:17:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #32/50 start                                                    
2025-07-23 21:17:30: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #33/50 start                                                    
2025-07-23 21:17:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #34/50 start                                                    
2025-07-23 21:18:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #35/50 start                                                    
2025-07-23 21:18:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #36/50 start                                                    
2025-07-23 21:18:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #37/50 start                                                    
2025-07-23 21:18:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #38/50 start                                                    
2025-07-23 21:19:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #39/50 start                                                    
2025-07-23 21:19:26: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #40/50 start                                                    
2025-07-23 21:19:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #41/50 start                                                    
2025-07-23 21:19:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #42/50 start                                                    
2025-07-23 21:20:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #43/50 start                                                    
2025-07-23 21:20:28: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #44/50 start                                                    
2025-07-23 21:20:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #45/50 start                                                    
2025-07-23 21:20:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #46/50 start                                                    
2025-07-23 21:21:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #47/50 start                                                    
2025-07-23 21:21:28: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #48/50 start                                                    
2025-07-23 21:21:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #49/50 start                                                    
2025-07-23 21:21:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, eval #50/50 start                                                    
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:22:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 21:24:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, unknown 1 = ∑1/50, started new job                                   
2025-07-23 21:24:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/1 = ∑4/50, started new job                         
2025-07-23 21:24:14: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending 3 = ∑3/50, started new job                                   
2025-07-23 21:24:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/2 = ∑5/50, started new job                         
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/9 = ∑12/50, started new job                        
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/9 = ∑12/50, started new job                        
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/9 = ∑12/50, started new job                        
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/8 = ∑11/50, started new job                        
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/12 = ∑15/50, started new job                       
2025-07-23 21:24:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/10 = ∑13/50, started new job                       
2025-07-23 21:24:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:24:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:24:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:24:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:24:19: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:24:19: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, pending/unknown 3/13 = ∑16/50, started new job                       
2025-07-23 21:26:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 21:26:10: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 21:26:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 21:26:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/unknown 16/1 = ∑17/50, started new job                       
2025-07-23 21:26:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 17 = ∑17/50, starting new job                                
2025-07-23 21:26:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 17 = ∑17/50, starting new job                                
2025-07-23 21:26:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 17 = ∑17/50, starting new job                                
2025-07-23 21:26:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 17 = ∑17/50, starting new job                                
2025-07-23 21:26:50: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/unknown 17/1 = ∑18/50, started new job                       
2025-07-23 21:26:52: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:26:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:26:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:26:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:26:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:26:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 17/1 = ∑18/50, starting new job                      
2025-07-23 21:27:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 17/1/1 = ∑19/50, started new job             
2025-07-23 21:28:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/2/1 = ∑22/50, started new job             
2025-07-23 21:28:37: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/2/2 = ∑23/50, started new job             
2025-07-23 21:28:37: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/2/2 = ∑23/50, started new job             
2025-07-23 21:28:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/2/2 = ∑23/50, started new job             
2025-07-23 21:28:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/2 = ∑25/50, starting new job            
2025-07-23 21:28:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/3 = ∑26/50, started new job             
2025-07-23 21:28:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/3 = ∑26/50, started new job             
2025-07-23 21:28:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/3 = ∑26/50, started new job             
2025-07-23 21:28:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/3 = ∑26/50, starting new job            
2025-07-23 21:28:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/4 = ∑27/50, starting new job            
2025-07-23 21:28:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/4/4 = ∑27/50, started new job             
2025-07-23 21:28:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 19/8 = ∑27/50, starting new job                      
2025-07-23 21:28:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending 19/8 = ∑27/50, starting new job                      
2025-07-23 21:28:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/8/2 = ∑29/50, started new job             
2025-07-23 21:28:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/pending/unknown 19/8/2 = ∑29/50, started new job             
2025-07-23 21:29:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 29 = ∑29/50, starting new job                                
2025-07-23 21:30:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 28/1/2 = ∑31/50, started new job           
2025-07-23 21:30:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 28/1/2 = ∑31/50, started new job           
2025-07-23 21:30:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 30/1/2 = ∑33/50, started new job           
2025-07-23 21:30:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 30/1/1/3 = ∑35/50, started new job 
2025-07-23 21:30:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 30/1/1/3 = ∑35/50, started new job 
2025-07-23 21:30:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 30/1/1/3 = ∑35/50, starting new job
2025-07-23 21:30:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 30/1/1/3 = ∑35/50, started new job 
2025-07-23 21:30:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:30:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:31:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:31:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:31:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:31:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:31:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 30/1/4 = ∑35/50, starting new job          
2025-07-23 21:32:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed 34/1 = ∑35/50, starting new job                    
2025-07-23 21:32:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed 34/1 = ∑35/50, starting new job                    
2025-07-23 21:32:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed 34/1 = ∑35/50, starting new job                    
2025-07-23 21:32:50: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 34/1/1 = ∑36/50, started new job           
2025-07-23 21:32:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/1/2 = ∑38/50, started new job 
2025-07-23 21:32:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending 34/1/3 = ∑38/50, started new job           
2025-07-23 21:32:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/1 = ∑39/50, starting new job
2025-07-23 21:33:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/2 = ∑40/50, starting new job
2025-07-23 21:33:01: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/2 = ∑40/50, starting new job
2025-07-23 21:33:01: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/2 = ∑40/50, started new job 
2025-07-23 21:33:01: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/3 = ∑41/50, started new job 
2025-07-23 21:33:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/3/2 = ∑40/50, starting new job
2025-07-23 21:33:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/7/1 = ∑43/50, started new job 
2025-07-23 21:33:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/7/1 = ∑43/50, started new job 
2025-07-23 21:33:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 34/1/7/1 = ∑43/50, started new job 
2025-07-23 21:34:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 42/1/1 = ∑44/50, started new job           
2025-07-23 21:34:24: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed 44/1 = ∑45/50, started new job                     
2025-07-23 21:34:35: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 44/1/1 = ∑46/50, started new job           
2025-07-23 21:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/pending/unknown 44/1/1/1 = ∑47/50, started new job 
2025-07-23 21:35:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 46/1/3 = ∑50/50, started new job           
2025-07-23 21:35:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 46/1/3 = ∑50/50, started new job           
2025-07-23 21:35:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed/unknown 46/1/3 = ∑50/50, started new job           
2025-07-23 21:36:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running/completed 49/1 = ∑50/50, new result: 95.15                   
2025-07-23 21:36:52: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, finishing jobs, finished 1 job                  
2025-07-23 21:37:10: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:37:32: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:37:53: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:38:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:38:35: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:38:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:39:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:39:41: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:40:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:40:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:40:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:41:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:41:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:41:53: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:42:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:42:35: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:42:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:43:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:43:30: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:43:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:44:08: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:44:27: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:44:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:45:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:45:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:45:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:46:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:46:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:46:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:47:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:47:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:47:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:48:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:48:24: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:48:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:49:06: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:49:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:49:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:50:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:50:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:50:45: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:51:05: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:51:24: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:51:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:52:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:52:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:52:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:53:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:53:26: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:53:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:54:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:54:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:54:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:55:16: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:55:37: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:55:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:56:21: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:56:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:57:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:57:26: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:57:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:58:08: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:58:29: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:58:52: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:59:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:59:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 21:59:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:00:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:00:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:00:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:01:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:01:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:02:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:02:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:02:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:03:09: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:03:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:03:53: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:04:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:04:36: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:04:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:05:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:05:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:06:03: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:06:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:06:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:07:07: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:07:29: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:07:50: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:08:12: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:08:34: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:08:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:09:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:09:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:10:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:10:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:10:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:11:06: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:11:28: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:11:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:12:10: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:12:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:12:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:13:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:13:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:14:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:14:22: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:14:44: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:15:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:15:26: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:15:48: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:16:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:16:33: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:16:54: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:17:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:17:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:18:00: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:18:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:18:40: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                             
2025-07-23 22:18:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 49 = ∑49/50, new result: 97.09                               
2025-07-23 22:19:46: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job             
2025-07-23 22:20:01: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:20:22: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:20:43: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:21:06: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:21:27: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:21:49: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:22:11: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                             
2025-07-23 22:22:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 48 = ∑48/50, new result: 6.78                                
2025-07-23 22:23:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job             
2025-07-23 22:23:32: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:23:51: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:24:13: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:24:35: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:24:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:25:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:25:41: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:26:02: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                             
2025-07-23 22:26:23: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 47 = ∑47/50, new result: 98.31                               
2025-07-23 22:27:07: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job             
2025-07-23 22:27:22: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:27:42: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:28:04: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:28:25: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:28:47: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:29:09: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:29:31: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:29:52: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:30:15: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:30:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:30:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:31:17: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:31:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:31:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:32:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:32:38: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:32:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:33:18: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:33:39: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:33:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:34:20: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                             
2025-07-23 22:34:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:55: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:56: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:57: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:58: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:34:59: BoTorchGenerator, failed: 122, best VAL_ACC: 98.44, timeout/running 41/5 = ∑46/50, job_failed                            
2025-07-23 22:39:09: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:10: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:10: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:10: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:10: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:10: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:11: BoTorchGenerator, failed: 129, best VAL_ACC: 98.44, timeout/running 34/5 = ∑39/50, job_failed                            
2025-07-23 22:39:34: BoTorchGenerator, failed: 154, best VAL_ACC: 98.44, timeout/running 9/5 = ∑14/50, job_failed                             
2025-07-23 22:39:36: BoTorchGenerator, failed: 154, best VAL_ACC: 98.44, timeout/running 9/5 = ∑14/50, job_failed                             
2025-07-23 22:40:57: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 46 jobs, finished 41 jobs             
2025-07-23 22:41:11: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:41:31: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:41:52: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:42:12: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:42:32: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:42:52: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:43:12: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:43:32: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:43:53: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:44:13: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:44:32: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, running 5 = ∑5/50, waiting for 5 jobs                                
2025-07-23 22:45:05: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, timeout 5 = ∑5/50, job_failed                                        
2025-07-23 22:45:06: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, timeout 5 = ∑5/50, job_failed                                        
2025-07-23 22:45:06: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, timeout 5 = ∑5/50, job_failed                                        
2025-07-23 22:45:07: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, timeout 5 = ∑5/50, job_failed                                        
2025-07-23 22:45:07: BoTorchGenerator, failed: 163, best VAL_ACC: 98.44, timeout 5 = ∑5/50, job_failed                                        
2025-07-23 22:46:02: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, waiting for 5 jobs, finished 5 jobs                                  
2025-07-23 22:54:35: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #1/50                                             
2025-07-23 22:55:01: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #2/50                                             
2025-07-23 22:55:27: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #3/50                                             
2025-07-23 22:55:58: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #4/50                                             
2025-07-23 22:56:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #4/50                                             
2025-07-23 22:56:50: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #5/50                                             
2025-07-23 22:57:07: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #5/50                                             
2025-07-23 22:57:41: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #6/50                                             
2025-07-23 22:58:17: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #7/50                                             
2025-07-23 22:58:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #7/50                                             
2025-07-23 22:58:50: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #7/50                                             
2025-07-23 22:59:28: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #8/50                                             
2025-07-23 23:00:03: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #9/50                                             
2025-07-23 23:00:38: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #10/50                                            
2025-07-23 23:00:55: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #10/50                                            
2025-07-23 23:01:30: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #11/50                                            
2025-07-23 23:02:05: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #12/50                                            
2025-07-23 23:02:41: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #13/50                                            
2025-07-23 23:03:17: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #14/50                                            
2025-07-23 23:03:54: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #15/50                                            
2025-07-23 23:04:31: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #16/50                                            
2025-07-23 23:05:07: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #17/50                                            
2025-07-23 23:05:42: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #18/50                                            
2025-07-23 23:06:16: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #19/50                                            
2025-07-23 23:06:33: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #19/50                                            
2025-07-23 23:07:07: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #20/50                                            
2025-07-23 23:07:42: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #21/50                                            
2025-07-23 23:08:17: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #22/50                                            
2025-07-23 23:08:52: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #23/50                                            
2025-07-23 23:09:30: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #24/50                                            
2025-07-23 23:10:08: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #25/50                                            
2025-07-23 23:10:25: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #25/50                                            
2025-07-23 23:11:01: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #26/50                                            
2025-07-23 23:11:39: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #27/50                                            
2025-07-23 23:12:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #28/50                                            
2025-07-23 23:12:51: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #29/50                                            
2025-07-23 23:13:28: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #30/50                                            
2025-07-23 23:14:04: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #31/50                                            
2025-07-23 23:14:40: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #32/50                                            
2025-07-23 23:15:16: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #33/50                                            
2025-07-23 23:15:52: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #34/50                                            
2025-07-23 23:16:28: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #35/50                                            
2025-07-23 23:17:07: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #36/50                                            
2025-07-23 23:17:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #37/50                                            
2025-07-23 23:18:22: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #38/50                                            
2025-07-23 23:19:00: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #39/50                                            
2025-07-23 23:19:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #40/50                                            
2025-07-23 23:20:15: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #41/50                                            
2025-07-23 23:20:53: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #42/50                                            
2025-07-23 23:21:31: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #43/50                                            
2025-07-23 23:23:55: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #44/50                                            
2025-07-23 23:24:33: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #45/50                                            
2025-07-23 23:25:11: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #46/50                                            
2025-07-23 23:25:51: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #47/50                                            
2025-07-23 23:26:08: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #47/50                                            
2025-07-23 23:26:48: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #48/50                                            
2025-07-23 23:27:06: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #48/50                                            
2025-07-23 23:28:13: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #49/50                                            
2025-07-23 23:28:51: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, getting new HP set #50/50                                            
2025-07-23 23:29:28: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, requested 50 jobs, got 50, 51.77 s/job                               
2025-07-23 23:29:47: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #1/50 start                                                     
2025-07-23 23:30:08: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #2/50 start                                                     
2025-07-23 23:30:29: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #3/50 start                                                     
2025-07-23 23:30:50: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #4/50 start                                                     
2025-07-23 23:31:11: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #5/50 start                                                     
2025-07-23 23:31:31: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #6/50 start                                                     
2025-07-23 23:31:52: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #7/50 start                                                     
2025-07-23 23:32:13: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #8/50 start                                                     
2025-07-23 23:32:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #9/50 start                                                     
2025-07-23 23:32:55: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #10/50 start                                                    
2025-07-23 23:33:16: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #11/50 start                                                    
2025-07-23 23:33:36: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #12/50 start                                                    
2025-07-23 23:33:58: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #13/50 start                                                    
2025-07-23 23:34:19: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #14/50 start                                                    
2025-07-23 23:35:06: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #15/50 start                                                    
2025-07-23 23:35:27: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #16/50 start                                                    
2025-07-23 23:35:47: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #17/50 start                                                    
2025-07-23 23:36:08: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #18/50 start                                                    
2025-07-23 23:36:29: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #19/50 start                                                    
2025-07-23 23:36:50: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #20/50 start                                                    
2025-07-23 23:37:11: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #21/50 start                                                    
2025-07-23 23:37:32: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #22/50 start                                                    
2025-07-23 23:37:52: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #23/50 start                                                    
2025-07-23 23:38:13: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #24/50 start                                                    
2025-07-23 23:38:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #25/50 start                                                    
2025-07-23 23:38:55: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #26/50 start                                                    
2025-07-23 23:39:15: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #27/50 start                                                    
2025-07-23 23:39:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #28/50 start                                                    
2025-07-23 23:39:59: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #29/50 start                                                    
2025-07-23 23:40:20: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #30/50 start                                                    
2025-07-23 23:40:41: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #31/50 start                                                    
2025-07-23 23:41:01: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #32/50 start                                                    
2025-07-23 23:41:21: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #33/50 start                                                    
2025-07-23 23:41:42: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #34/50 start                                                    
2025-07-23 23:42:02: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #35/50 start                                                    
2025-07-23 23:42:23: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #36/50 start                                                    
2025-07-23 23:42:44: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #37/50 start                                                    
2025-07-23 23:43:05: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #38/50 start                                                    
2025-07-23 23:43:25: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #39/50 start                                                    
2025-07-23 23:43:48: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #40/50 start                                                    
2025-07-23 23:44:09: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #41/50 start                                                    
2025-07-23 23:44:30: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #42/50 start                                                    
2025-07-23 23:44:53: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #43/50 start                                                    
2025-07-23 23:45:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #44/50 start                                                    
2025-07-23 23:45:35: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #45/50 start                                                    
2025-07-23 23:45:58: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #46/50 start                                                    
2025-07-23 23:46:19: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #47/50 start                                                    
2025-07-23 23:46:40: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #48/50 start                                                    
2025-07-23 23:47:01: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #49/50 start                                                    
2025-07-23 23:47:22: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, eval #50/50 start                                                    
2025-07-23 23:47:44: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:44: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:44: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:44: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:47:45: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, starting new job                                                     
2025-07-23 23:50:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending 2 = ∑2/50, started new job                                   
2025-07-23 23:50:36: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/3 = ∑5/50, started new job                         
2025-07-23 23:50:36: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/6 = ∑8/50, started new job                         
2025-07-23 23:50:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/8 = ∑10/50, started new job                        
2025-07-23 23:50:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/10 = ∑12/50, started new job                       
2025-07-23 23:50:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/10 = ∑12/50, started new job                       
2025-07-23 23:50:37: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/10 = ∑12/50, started new job                       
2025-07-23 23:50:38: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/11 = ∑13/50, started new job                       
2025-07-23 23:50:38: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/11 = ∑13/50, started new job                       
2025-07-23 23:50:39: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/11 = ∑13/50, started new job                       
2025-07-23 23:50:39: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/11 = ∑13/50, started new job                       
2025-07-23 23:50:39: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 2/11 = ∑13/50, started new job                       
2025-07-23 23:50:39: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending 14 = ∑14/50, started new job                                 
2025-07-23 23:50:40: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 14/2 = ∑16/50, started new job                       
2025-07-23 23:50:41: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 14/2 = ∑16/50, started new job                       
2025-07-23 23:50:41: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, pending/unknown 14/2 = ∑16/50, started new job                       
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:14: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:15: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:17: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:17: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:19: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:19: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:25: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:27: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:54:28: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                
2025-07-23 23:57:18: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 16/1 = ∑17/50, started new job                       
2025-07-23 23:57:20: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 16/2 = ∑18/50, started new job                       
2025-07-23 23:57:30: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 18/4 = ∑22/50, started new job                       
2025-07-23 23:57:30: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 18/3 = ∑21/50, started new job                       
2025-07-23 23:57:31: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 18/5 = ∑23/50, started new job                       
2025-07-23 23:57:32: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/unknown 18/8 = ∑26/50, started new job                       
2025-07-23 23:57:33: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/2 = ∑30/50, started new job            
2025-07-23 23:57:33: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/3 = ∑31/50, started new job            
2025-07-23 23:57:33: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/3 = ∑31/50, started new job            
2025-07-23 23:57:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/3 = ∑31/50, started new job            
2025-07-23 23:57:34: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-23 23:57:35: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-23 23:57:35: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-23 23:57:35: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-23 23:57:36: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-23 23:57:36: BoTorchGenerator, failed: 168, best VAL_ACC: 98.44, running/pending/unknown 18/10/4 = ∑32/50, started new job            
2025-07-24 00:00:06: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-24 00:00:52: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job
2025-07-24 00:01:38: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/1 = ∑33/50, started new job
2025-07-24 00:01:40: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/1 = ∑33/50, starting new job
2025-07-24 00:01:42: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 31/1/1 = ∑33/50, starting new job
2025-07-24 00:01:45: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 31/1/1 = ∑33/50, starting new job
2025-07-24 00:01:46: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 31/1/1 = ∑33/50, starting new job
2025-07-24 00:01:56: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:01:59: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:00: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:02: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:03: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:04: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:04: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:05: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:05: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:05: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 32/1 = ∑33/50, starting new job          
2025-07-24 00:02:10: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 32/1/1 = ∑34/50, started new job 
2025-07-24 00:04:57: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 33/1/2 = ∑36/50, started new job 
2025-07-24 00:04:58: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 33/1/2 = ∑36/50, started new job 
2025-07-24 00:05:02: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 33/1/2/2 = ∑38/50, started new job
2025-07-24 00:05:04: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 33/1/2/2 = ∑38/50, started new job
2025-07-24 00:05:14: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 37/1/2 = ∑40/50, started new job          
2025-07-24 00:05:15: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 37/1/2 = ∑40/50, started new job          
2025-07-24 00:05:18: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/2/2 = ∑42/50, started new job
2025-07-24 00:05:20: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/2 = ∑46/50, starting new job
2025-07-24 00:05:20: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/2 = ∑46/50, started new job 
2025-07-24 00:05:21: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/2 = ∑46/50, started new job 
2025-07-24 00:05:21: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/4 = ∑48/50, started new job 
2025-07-24 00:05:21: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/4 = ∑48/50, started new job 
2025-07-24 00:05:21: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/4 = ∑48/50, started new job 
2025-07-24 00:05:22: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/4 = ∑48/50, started new job 
2025-07-24 00:05:23: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending/unknown 36/1/1/6/4 = ∑48/50, started new job 
2025-07-24 00:05:40: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory/pending 36/1/1/10 = ∑48/50, starting new job         
2025-07-24 00:08:58: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory 48/1/1 = ∑50/50, started new job                     
2025-07-24 00:08:58: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory 48/1/1 = ∑50/50, started new job                     
2025-07-24 00:10:36: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory 47/2/1 = ∑50/50, job_failed                          
2025-07-24 00:10:36: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory 47/2/1 = ∑50/50, new result: 95.0                    
2025-07-24 00:10:37: BoTorchGenerator, failed: 168 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed/out_of_memory 47/2/1 = ∑50/50, new result: 11.35                   
2025-07-24 00:12:21: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed 46/1 = ∑47/50, finishing jobs, finished 3 jobs                     
2025-07-24 00:12:44: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/completed 46/1 = ∑47/50, new result: 10.1                                    
2025-07-24 00:13:57: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, finishing previous jobs (47), finished 1 job                    
2025-07-24 00:14:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:14:49: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:15:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:15:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:16:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:16:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:17:17: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:17:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:18:17: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:18:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:19:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:19:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:20:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:20:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:21:16: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:21:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:22:17: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:22:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:23:16: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:23:46: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:24:16: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:24:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:25:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:25:50: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:26:20: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:26:49: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:27:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:27:49: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:28:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:28:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:29:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:29:49: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:30:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:30:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:31:16: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:31:43: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:32:10: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 46 jobs                                             
2025-07-24 00:32:35: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, new result: 98.27                                               
2025-07-24 00:33:38: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 46 jobs, finished 1 job                             
2025-07-24 00:33:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:34:25: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:34:52: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:35:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:35:46: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:36:13: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:36:41: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:37:08: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:37:35: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:38:02: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:38:29: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:38:57: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:39:25: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:39:53: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:40:19: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:40:47: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:41:14: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:41:41: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:42:09: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:42:36: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:43:02: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:43:31: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:43:57: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:44:26: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, waiting for 45 jobs                                             
2025-07-24 00:44:52: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 45 = ∑45/50, new result: 98.05                                               
2025-07-24 00:46:01: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, waiting for 45 jobs, finished 1 job                             
2025-07-24 00:46:21: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, waiting for 44 jobs                                             
2025-07-24 00:46:49: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, waiting for 44 jobs                                             
2025-07-24 00:47:18: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, waiting for 44 jobs                                             
2025-07-24 00:47:46: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, waiting for 44 jobs                                             
2025-07-24 00:48:11: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 44 = ∑44/50, new result: 97.63                                               
2025-07-24 00:49:16: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 44 jobs, finished 1 job                             
2025-07-24 00:49:43: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:50:11: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:50:39: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:51:08: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:51:36: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:52:12: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:52:39: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:53:07: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:53:48: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:54:15: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 43 = ∑43/50, waiting for 43 jobs                                             
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:54:58: BoTorchGenerator, failed: 169 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 14/29 = ∑43/50, job_failed                                           
2025-07-24 00:59:28: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 43 jobs, finished 14 jobs                           
2025-07-24 00:59:49: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:00:17: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:00:44: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:01:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:01:39: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:02:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:02:35: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:03:03: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:03:39: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:04:07: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 01:04:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:05:12: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:05:42: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:06:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:06:39: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:07:06: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:07:33: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:08:02: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:08:29: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:08:56: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:09:24: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:09:51: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:10:20: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:10:49: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:11:18: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:11:45: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:12:13: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:12:41: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:13:08: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:13:35: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:14:02: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:14:29: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:10: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:11: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:15:12: BoTorchGenerator, failed: 183 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 14 = ∑14/50, job_failed                                                      
2025-07-24 01:18:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, waiting for 14 jobs, finished 14 jobs                                                
2025-07-24 01:31:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #1/50                                                             
2025-07-24 01:32:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #2/50                                                             
2025-07-24 01:32:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #3/50                                                             
2025-07-24 01:33:24: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #4/50                                                             
2025-07-24 01:34:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #5/50                                                             
2025-07-24 01:34:44: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #6/50                                                             
2025-07-24 01:35:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #6/50                                                             
2025-07-24 01:35:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #7/50                                                             
2025-07-24 01:36:23: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #8/50                                                             
2025-07-24 01:37:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #9/50                                                             
2025-07-24 01:37:42: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 01:38:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #11/50                                                            
2025-07-24 01:39:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #12/50                                                            
2025-07-24 01:39:20: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #12/50                                                            
2025-07-24 01:39:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 01:40:17: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 01:40:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #14/50                                                            
2025-07-24 01:41:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #15/50                                                            
2025-07-24 01:42:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #16/50                                                            
2025-07-24 01:42:52: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 01:43:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #18/50                                                            
2025-07-24 01:44:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #19/50                                                            
2025-07-24 01:44:44: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #20/50                                                            
2025-07-24 01:45:22: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #21/50                                                            
2025-07-24 01:45:59: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #22/50                                                            
2025-07-24 01:46:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #23/50                                                            
2025-07-24 01:47:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #24/50                                                            
2025-07-24 01:47:53: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #25/50                                                            
2025-07-24 01:48:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #26/50                                                            
2025-07-24 01:49:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #27/50                                                            
2025-07-24 01:49:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #28/50                                                            
2025-07-24 01:50:27: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #29/50                                                            
2025-07-24 01:51:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #30/50                                                            
2025-07-24 01:51:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #31/50                                                            
2025-07-24 01:52:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #32/50                                                            
2025-07-24 01:52:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #33/50                                                            
2025-07-24 01:53:34: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #34/50                                                            
2025-07-24 01:54:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #35/50                                                            
2025-07-24 01:54:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #36/50                                                            
2025-07-24 01:55:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 01:56:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #38/50                                                            
2025-07-24 01:56:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #39/50                                                            
2025-07-24 01:57:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #40/50                                                            
2025-07-24 01:58:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #41/50                                                            
2025-07-24 01:58:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #42/50                                                            
2025-07-24 01:59:09: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #43/50                                                            
2025-07-24 01:59:41: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #44/50                                                            
2025-07-24 02:00:17: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #45/50                                                            
2025-07-24 02:00:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #45/50                                                            
2025-07-24 02:01:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #46/50                                                            
2025-07-24 02:02:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #47/50                                                            
2025-07-24 02:03:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #48/50                                                            
2025-07-24 02:03:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #49/50                                                            
2025-07-24 02:04:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 02:05:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 02:05:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, requested 50 jobs, got 50, 56.38 s/job                                               
2025-07-24 02:05:52: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #1/50 start                                                                     
2025-07-24 02:06:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #2/50 start                                                                     
2025-07-24 02:06:27: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #3/50 start                                                                     
2025-07-24 02:06:46: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #4/50 start                                                                     
2025-07-24 02:07:03: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #5/50 start                                                                     
2025-07-24 02:07:19: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #6/50 start                                                                     
2025-07-24 02:07:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #7/50 start                                                                     
2025-07-24 02:07:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #8/50 start                                                                     
2025-07-24 02:08:15: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #9/50 start                                                                     
2025-07-24 02:08:32: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #10/50 start                                                                    
2025-07-24 02:08:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #11/50 start                                                                    
2025-07-24 02:09:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #12/50 start                                                                    
2025-07-24 02:09:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #13/50 start                                                                    
2025-07-24 02:09:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #14/50 start                                                                    
2025-07-24 02:09:54: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #15/50 start                                                                    
2025-07-24 02:10:09: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #16/50 start                                                                    
2025-07-24 02:10:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #17/50 start                                                                    
2025-07-24 02:10:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #18/50 start                                                                    
2025-07-24 02:11:00: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #19/50 start                                                                    
2025-07-24 02:11:16: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #20/50 start                                                                    
2025-07-24 02:11:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #21/50 start                                                                    
2025-07-24 02:11:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #22/50 start                                                                    
2025-07-24 02:12:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #23/50 start                                                                    
2025-07-24 02:12:23: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #24/50 start                                                                    
2025-07-24 02:12:39: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #25/50 start                                                                    
2025-07-24 02:12:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #26/50 start                                                                    
2025-07-24 02:13:15: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #27/50 start                                                                    
2025-07-24 02:13:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #28/50 start                                                                    
2025-07-24 02:13:46: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #29/50 start                                                                    
2025-07-24 02:14:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #30/50 start                                                                    
2025-07-24 02:14:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #31/50 start                                                                    
2025-07-24 02:14:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #32/50 start                                                                    
2025-07-24 02:14:54: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #33/50 start                                                                    
2025-07-24 02:15:11: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #34/50 start                                                                    
2025-07-24 02:15:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #35/50 start                                                                    
2025-07-24 02:15:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #36/50 start                                                                    
2025-07-24 02:16:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #37/50 start                                                                    
2025-07-24 02:16:17: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #38/50 start                                                                    
2025-07-24 02:16:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #39/50 start                                                                    
2025-07-24 02:16:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #40/50 start                                                                    
2025-07-24 02:17:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #41/50 start                                                                    
2025-07-24 02:17:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #42/50 start                                                                    
2025-07-24 02:17:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #43/50 start                                                                    
2025-07-24 02:17:52: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #44/50 start                                                                    
2025-07-24 02:18:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #45/50 start                                                                    
2025-07-24 02:18:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #46/50 start                                                                    
2025-07-24 02:18:40: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #47/50 start                                                                    
2025-07-24 02:18:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #48/50 start                                                                    
2025-07-24 02:19:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #49/50 start                                                                    
2025-07-24 02:19:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #50/50 start                                                                    
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:19:46: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 02:21:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 4 = ∑4/50, started new job                                                   
2025-07-24 02:21:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 02:21:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 02:21:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 02:21:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 02:21:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 02:22:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/2 = ∑8/50, started new job                                         
2025-07-24 02:22:03: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/3 = ∑9/50, started new job                                         
2025-07-24 02:22:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 11 = ∑11/50, started new job                                                 
2025-07-24 02:22:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 11 = ∑11/50, started new job                                                 
2025-07-24 02:22:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:22:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:22:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:22:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:22:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:22:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 11/5 = ∑16/50, started new job                                       
2025-07-24 02:24:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:39: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:39: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:41: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:42: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:24:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 02:26:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/3 = ∑19/50, started new job                                       
2025-07-24 02:26:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/3 = ∑19/50, started new job                                       
2025-07-24 02:26:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/3 = ∑19/50, started new job                                       
2025-07-24 02:27:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/3/1 = ∑20/50, started new job                             
2025-07-24 02:27:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/3/5 = ∑24/50, started new job                             
2025-07-24 02:27:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/3/5 = ∑24/50, started new job                             
2025-07-24 02:27:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/3/5 = ∑24/50, started new job                             
2025-07-24 02:27:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/8 = ∑24/50, started new job                                       
2025-07-24 02:27:09: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/6 = ∑30/50, started new job                             
2025-07-24 02:27:09: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/7 = ∑31/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:27:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/8 = ∑32/50, started new job                             
2025-07-24 02:29:04: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:29:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:29:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:13: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:17: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:18: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:18: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:18: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:18: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 02:30:23: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 32/1 = ∑33/50, started new job                                       
2025-07-24 02:30:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 32/1/1 = ∑34/50, started new job                             
2025-07-24 02:30:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 32/1/2 = ∑35/50, started new job                             
2025-07-24 02:32:27: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 35/2 = ∑37/50, started new job                                       
2025-07-24 02:32:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 35/2 = ∑37/50, started new job                                       
2025-07-24 02:32:32: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 35/2/2 = ∑39/50, started new job                             
2025-07-24 02:32:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 37/2 = ∑39/50, started new job                                       
2025-07-24 02:32:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 39/1 = ∑40/50, started new job                                       
2025-07-24 02:32:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 39/3 = ∑42/50, started new job                                       
2025-07-24 02:32:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 39/3 = ∑42/50, started new job                                       
2025-07-24 02:32:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 39/4 = ∑43/50, started new job                                       
2025-07-24 02:32:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 39/4/1 = ∑44/50, started new job                             
2025-07-24 02:32:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 39/6/2 = ∑47/50, started new job                             
2025-07-24 02:32:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 39/6/3 = ∑48/50, started new job                             
2025-07-24 02:32:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 39/6/3 = ∑48/50, started new job                             
2025-07-24 02:32:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 39/6/3 = ∑48/50, started new job                             
2025-07-24 02:33:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, starting new job                                                
2025-07-24 02:33:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, starting new job                                                
2025-07-24 02:35:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 48/2 = ∑50/50, started new job                                       
2025-07-24 02:35:22: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 48/2 = ∑50/50, started new job                                       
2025-07-24 02:36:00: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 02:36:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 02:36:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 02:37:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 02:37:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 02:37:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, new result: 96.21                                               
2025-07-24 02:38:54: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job                             
2025-07-24 02:39:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:39:34: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:39:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:40:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:40:46: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:41:10: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:41:34: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:41:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:42:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:42:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:43:11: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:43:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:43:59: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:44:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:44:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:45:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:45:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:45:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:46:19: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:46:44: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:47:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:47:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:47:59: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:48:23: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:48:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:49:13: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:49:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:49:59: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:50:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:50:48: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:51:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:51:38: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:52:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:52:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:52:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:53:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:53:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:54:00: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:54:24: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:54:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:55:14: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:55:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:56:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:56:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:56:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:57:13: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:57:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:58:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:58:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:58:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:59:11: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:59:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 02:59:59: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:00:24: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:00:48: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:01:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:01:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:02:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:02:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:02:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:03:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:03:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:03:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:04:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:04:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:05:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:05:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:06:01: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:06:24: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:06:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:07:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:07:36: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:08:03: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:08:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:08:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:09:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:09:34: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:09:56: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:10:19: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:10:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:11:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:11:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 03:11:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, new result: 98.26                                               
2025-07-24 03:12:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job                             
2025-07-24 03:13:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:13:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:13:49: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:14:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:14:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:15:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:15:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:15:51: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:16:17: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:16:42: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 03:17:05: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, new result: 87.71                                               
2025-07-24 03:17:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 48 jobs, finished 1 job                             
2025-07-24 03:18:13: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:18:39: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:19:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:19:26: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:19:50: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:20:12: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:20:37: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:21:02: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:21:25: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:21:51: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:22:16: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:22:40: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:23:06: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:23:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:23:53: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:24:18: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:24:43: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:25:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:25:33: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:25:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:26:22: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:26:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:27:07: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:27:32: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:27:57: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:28:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:28:45: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:29:08: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:29:35: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:29:58: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:30:21: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:30:47: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 03:31:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:28: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:29: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:30: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:31:31: BoTorchGenerator, failed: 197 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 35/12 = ∑47/50, job_failed                                           
2025-07-24 03:36:54: BoTorchGenerator, failed: 209 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 23/12 = ∑35/50, job_failed                                           
2025-07-24 03:36:54: BoTorchGenerator, failed: 209 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 23/12 = ∑35/50, job_failed                                           
2025-07-24 03:36:54: BoTorchGenerator, failed: 209 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 23/12 = ∑35/50, job_failed                                           
2025-07-24 03:38:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 47 jobs, finished 35 jobs                           
2025-07-24 03:39:00: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             
2025-07-24 03:39:29: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             
2025-07-24 03:40:00: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             
2025-07-24 03:40:28: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             
2025-07-24 03:40:58: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 12 = ∑12/50, waiting for 12 jobs                                             
2025-07-24 03:41:39: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:39: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:41:40: BoTorchGenerator, failed: 232 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 12 = ∑12/50, job_failed                                                      
2025-07-24 03:44:42: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, waiting for 12 jobs, finished 12 jobs                                                
2025-07-24 03:58:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #1/50                                                             
2025-07-24 03:59:29: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #2/50                                                             
2025-07-24 04:00:13: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #3/50                                                             
2025-07-24 04:00:58: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #4/50                                                             
2025-07-24 04:01:19: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #4/50                                                             
2025-07-24 04:02:02: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #5/50                                                             
2025-07-24 04:02:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #6/50                                                             
2025-07-24 04:03:28: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #7/50                                                             
2025-07-24 04:04:16: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #8/50                                                             
2025-07-24 04:05:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #9/50                                                             
2025-07-24 04:05:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 04:06:05: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 04:06:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 04:06:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 04:07:31: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #11/50                                                            
2025-07-24 04:08:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #12/50                                                            
2025-07-24 04:08:57: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 04:09:41: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #14/50                                                            
2025-07-24 04:10:24: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #15/50                                                            
2025-07-24 04:11:08: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #16/50                                                            
2025-07-24 04:11:50: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 04:12:12: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 04:12:31: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 04:12:52: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 04:13:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #18/50                                                            
2025-07-24 04:14:18: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #19/50                                                            
2025-07-24 04:14:39: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #19/50                                                            
2025-07-24 04:15:21: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #20/50                                                            
2025-07-24 04:16:03: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #21/50                                                            
2025-07-24 04:16:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #22/50                                                            
2025-07-24 04:17:28: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #23/50                                                            
2025-07-24 04:18:11: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #24/50                                                            
2025-07-24 04:18:55: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #25/50                                                            
2025-07-24 04:19:38: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #26/50                                                            
2025-07-24 04:20:20: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #27/50                                                            
2025-07-24 04:21:03: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #28/50                                                            
2025-07-24 04:21:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #29/50                                                            
2025-07-24 04:22:30: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #30/50                                                            
2025-07-24 04:23:14: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #31/50                                                            
2025-07-24 04:24:00: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #32/50                                                            
2025-07-24 04:24:43: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #33/50                                                            
2025-07-24 04:25:27: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #34/50                                                            
2025-07-24 04:26:11: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #35/50                                                            
2025-07-24 04:26:55: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #36/50                                                            
2025-07-24 04:27:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #36/50                                                            
2025-07-24 04:27:59: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 04:28:41: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #38/50                                                            
2025-07-24 04:29:22: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #39/50                                                            
2025-07-24 04:30:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #40/50                                                            
2025-07-24 04:30:50: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #41/50                                                            
2025-07-24 04:33:32: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #42/50                                                            
2025-07-24 04:34:16: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #43/50                                                            
2025-07-24 04:35:00: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #44/50                                                            
2025-07-24 04:35:43: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #45/50                                                            
2025-07-24 04:36:28: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #46/50                                                            
2025-07-24 04:36:48: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #46/50                                                            
2025-07-24 04:37:33: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #47/50                                                            
2025-07-24 04:38:17: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #48/50                                                            
2025-07-24 04:39:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #49/50                                                            
2025-07-24 04:40:19: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 04:41:04: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, requested 50 jobs, got 50, 67.11 s/job                                               
2025-07-24 04:41:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #1/50 start                                                                     
2025-07-24 04:41:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #2/50 start                                                                     
2025-07-24 04:42:10: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #3/50 start                                                                     
2025-07-24 04:42:33: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #4/50 start                                                                     
2025-07-24 04:42:57: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #5/50 start                                                                     
2025-07-24 04:43:19: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #6/50 start                                                                     
2025-07-24 04:43:43: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #7/50 start                                                                     
2025-07-24 04:44:07: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #8/50 start                                                                     
2025-07-24 04:44:30: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #9/50 start                                                                     
2025-07-24 04:44:54: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #10/50 start                                                                    
2025-07-24 04:45:16: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #11/50 start                                                                    
2025-07-24 04:45:40: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #12/50 start                                                                    
2025-07-24 04:46:03: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #13/50 start                                                                    
2025-07-24 04:46:27: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #14/50 start                                                                    
2025-07-24 04:46:51: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #15/50 start                                                                    
2025-07-24 04:47:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #16/50 start                                                                    
2025-07-24 04:47:38: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #17/50 start                                                                    
2025-07-24 04:48:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #18/50 start                                                                    
2025-07-24 04:48:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #19/50 start                                                                    
2025-07-24 04:48:49: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #20/50 start                                                                    
2025-07-24 04:49:12: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #21/50 start                                                                    
2025-07-24 04:49:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #22/50 start                                                                    
2025-07-24 04:49:58: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #23/50 start                                                                    
2025-07-24 04:50:21: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #24/50 start                                                                    
2025-07-24 04:50:44: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #25/50 start                                                                    
2025-07-24 04:51:09: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #26/50 start                                                                    
2025-07-24 04:51:32: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #27/50 start                                                                    
2025-07-24 04:51:55: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #28/50 start                                                                    
2025-07-24 04:52:20: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #29/50 start                                                                    
2025-07-24 04:52:43: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #30/50 start                                                                    
2025-07-24 04:53:07: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #31/50 start                                                                    
2025-07-24 04:53:30: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #32/50 start                                                                    
2025-07-24 04:53:53: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #33/50 start                                                                    
2025-07-24 04:54:18: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #34/50 start                                                                    
2025-07-24 04:54:42: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #35/50 start                                                                    
2025-07-24 04:55:10: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #36/50 start                                                                    
2025-07-24 04:55:34: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #37/50 start                                                                    
2025-07-24 04:56:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #38/50 start                                                                    
2025-07-24 04:56:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #39/50 start                                                                    
2025-07-24 04:56:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #40/50 start                                                                    
2025-07-24 04:57:11: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #41/50 start                                                                    
2025-07-24 04:57:34: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #42/50 start                                                                    
2025-07-24 04:57:57: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #43/50 start                                                                    
2025-07-24 04:58:20: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #44/50 start                                                                    
2025-07-24 04:58:45: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #45/50 start                                                                    
2025-07-24 04:59:08: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #46/50 start                                                                    
2025-07-24 04:59:33: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #47/50 start                                                                    
2025-07-24 04:59:55: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #48/50 start                                                                    
2025-07-24 05:00:19: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #49/50 start                                                                    
2025-07-24 05:00:41: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #50/50 start                                                                    
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:06: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:01:07: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 05:04:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 5 = ∑5/50, started new job                                                   
2025-07-24 05:04:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 5 = ∑5/50, started new job                                                   
2025-07-24 05:04:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 6 = ∑6/50, started new job                                                   
2025-07-24 05:04:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 7 = ∑7/50, started new job                                                   
2025-07-24 05:04:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/1 = ∑7/50, started new job                                         
2025-07-24 05:04:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/3 = ∑9/50, started new job                                         
2025-07-24 05:04:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/3 = ∑9/50, started new job                                         
2025-07-24 05:04:27: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/3 = ∑9/50, started new job                                         
2025-07-24 05:04:27: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 6/3 = ∑9/50, started new job                                         
2025-07-24 05:04:32: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 10/1 = ∑11/50, started new job                                       
2025-07-24 05:04:33: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 10/4 = ∑14/50, started new job                                       
2025-07-24 05:04:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 10/6 = ∑16/50, started new job                                       
2025-07-24 05:04:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 10/6 = ∑16/50, started new job                                       
2025-07-24 05:04:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 10/6 = ∑16/50, started new job                                       
2025-07-24 05:04:36: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 16 = ∑16/50, started new job                                                 
2025-07-24 05:04:36: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 16 = ∑16/50, started new job                                                 
2025-07-24 05:08:39: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:39: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:39: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:39: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:40: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:40: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:41: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:41: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:43: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:45: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:48: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:50: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:08:54: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:11:53: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:12:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 15/1 = ∑16/50, starting new job                                
2025-07-24 05:12:18: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 15/1/1 = ∑17/50, started new job                       
2025-07-24 05:12:21: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/5 = ∑22/50, started new job             
2025-07-24 05:12:22: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/6 = ∑23/50, started new job             
2025-07-24 05:12:22: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/7 = ∑24/50, started new job             
2025-07-24 05:12:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/11 = ∑28/50, started new job            
2025-07-24 05:12:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/11 = ∑28/50, started new job            
2025-07-24 05:12:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/11 = ∑28/50, started new job            
2025-07-24 05:12:24: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/13 = ∑30/50, started new job            
2025-07-24 05:12:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/13 = ∑30/50, started new job            
2025-07-24 05:12:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/13 = ∑30/50, started new job            
2025-07-24 05:12:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/13 = ∑30/50, started new job            
2025-07-24 05:12:25: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 15/1/1/13 = ∑30/50, started new job            
2025-07-24 05:12:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 15/1/14 = ∑30/50, started new job                      
2025-07-24 05:12:26: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending 15/1/14 = ∑30/50, started new job                      
2025-07-24 05:13:49: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 29/1/1 = ∑31/50, started new job                       
2025-07-24 05:15:30: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 30/1/1 = ∑32/50, started new job                       
2025-07-24 05:16:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:16:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:16:46: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:16:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:16:47: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:01: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:03: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:11: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:13: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:14: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:14: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:14: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:14: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:17:15: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 31/1 = ∑32/50, starting new job                                
2025-07-24 05:20:22: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/3 = ∑35/50, started new job                       
2025-07-24 05:20:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/5 = ∑37/50, started new job                       
2025-07-24 05:20:23: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/5 = ∑37/50, started new job                       
2025-07-24 05:20:24: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/5 = ∑37/50, started new job                       
2025-07-24 05:20:24: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 31/1/5 = ∑37/50, started new job                       
2025-07-24 05:20:38: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 31/1/5/1 = ∑38/50, started new job             
2025-07-24 05:20:44: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 37/1/2 = ∑40/50, started new job                       
2025-07-24 05:20:45: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 37/1/2 = ∑40/50, started new job                       
2025-07-24 05:20:49: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/2/2 = ∑42/50, started new job             
2025-07-24 05:20:50: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/2/4 = ∑44/50, started new job             
2025-07-24 05:20:52: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/3 = ∑47/50, started new job             
2025-07-24 05:20:53: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/3 = ∑47/50, started new job             
2025-07-24 05:20:53: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/3 = ∑47/50, started new job             
2025-07-24 05:20:54: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/3 = ∑47/50, started new job             
2025-07-24 05:20:54: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/3 = ∑47/50, started new job             
2025-07-24 05:20:57: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/pending/unknown 37/1/6/4 = ∑48/50, started new job             
2025-07-24 05:24:59: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 47/1 = ∑48/50, starting new job                                
2025-07-24 05:24:59: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory 47/1 = ∑48/50, starting new job                                
2025-07-24 05:25:33: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 47/1/2 = ∑50/50, started new job                       
2025-07-24 05:25:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/unknown 47/1/2 = ∑50/50, started new job                       
2025-07-24 05:26:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/completed 48/1/1 = ∑50/50, job_failed                          
2025-07-24 05:26:35: BoTorchGenerator, failed: 244 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/out_of_memory/completed 48/1/1 = ∑50/50, new result: 95.77                   
2025-07-24 05:28:20: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, finishing jobs, finished 2 jobs                                 
2025-07-24 05:28:44: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, new result: 10.28                                               
2025-07-24 05:29:59: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, finishing previous jobs (48), finished 1 job                    
2025-07-24 05:30:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:30:54: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:31:25: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:31:58: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:32:29: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:33:00: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:33:29: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:34:01: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:34:32: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:35:04: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:35:35: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:36:07: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:36:39: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:37:09: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:37:40: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:38:13: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:38:45: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:39:17: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:39:48: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:40:19: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:40:49: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:41:21: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:41:54: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:42:26: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:42:58: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:43:29: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:44:02: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:44:33: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:45:04: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:45:36: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:46:07: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:46:39: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:47:10: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:47:41: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:48:11: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:48:42: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:49:14: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:49:47: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:50:17: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:50:49: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:51:20: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:51:51: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:52:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:52:56: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:53:27: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:54:00: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:54:30: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:55:01: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:55:33: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:56:04: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:56:35: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:57:08: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:57:40: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:58:11: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:58:43: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:59:15: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 05:59:47: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:00:19: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:00:51: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:01:23: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:01:54: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:02:25: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:02:57: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:03:30: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, waiting for 47 jobs                                             
2025-07-24 06:04:00: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 47 = ∑47/50, new result: 9.58                                                
2025-07-24 06:05:20: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 46 = ∑46/50, waiting for 47 jobs, finished 1 job                             
2025-07-24 06:05:45: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:06:18: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:06:50: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:07:22: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:07:54: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:08:28: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:09:00: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:09:33: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:10:03: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, waiting for 31 jobs                                             
2025-07-24 06:10:34: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 31 = ∑31/50, new result: 98.39                                               
2025-07-24 06:11:47: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 31 jobs, finished 1 job                             
2025-07-24 06:12:10: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:12:41: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:13:11: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:13:42: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:14:14: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:14:45: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:15:15: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 30 = ∑30/50, waiting for 30 jobs                                             
2025-07-24 06:15:47: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:16:20: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:16:55: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:17:28: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:17:59: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:18:30: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:19:02: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:19:33: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:20:05: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:20:36: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:21:05: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:21:39: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:22:10: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:22:43: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:23:12: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:23:43: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:24:13: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:24:47: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:25:25: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 17 = ∑17/50, waiting for 17 jobs                                             
2025-07-24 06:26:23: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:23: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:26:24: BoTorchGenerator, failed: 245 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 15/2 = ∑17/50, job_failed                                            
2025-07-24 06:30:55: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 17 jobs, finished 15 jobs                             
2025-07-24 06:31:16: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:31:47: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:32:18: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:32:49: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:33:20: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:33:52: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:34:21: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:34:51: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:35:18: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:35:48: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 2 = ∑2/50, waiting for 2 jobs                                                
2025-07-24 06:36:27: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 2 = ∑2/50, job_failed                                                        
2025-07-24 06:36:27: BoTorchGenerator, failed: 260 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout 2 = ∑2/50, job_failed                                                        
2025-07-24 06:37:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, waiting for 2 jobs, finished 2 jobs                                                  
2025-07-24 06:49:37: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #1/50                                                             
2025-07-24 06:50:16: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #2/50                                                             
2025-07-24 06:50:53: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #3/50                                                             
2025-07-24 06:51:33: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #4/50                                                             
2025-07-24 06:52:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #5/50                                                             
2025-07-24 06:52:53: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #6/50                                                             
2025-07-24 06:53:35: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #7/50                                                             
2025-07-24 06:54:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #8/50                                                             
2025-07-24 06:54:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #8/50                                                             
2025-07-24 06:55:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #9/50                                                             
2025-07-24 06:55:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #9/50                                                             
2025-07-24 06:56:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 06:56:49: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #11/50                                                            
2025-07-24 06:57:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #12/50                                                            
2025-07-24 06:58:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 06:58:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #14/50                                                            
2025-07-24 06:59:28: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #15/50                                                            
2025-07-24 07:00:05: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #16/50                                                            
2025-07-24 07:00:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 07:01:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #18/50                                                            
2025-07-24 07:01:42: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #18/50                                                            
2025-07-24 07:02:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #19/50                                                            
2025-07-24 07:03:00: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #20/50                                                            
2025-07-24 07:03:38: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #21/50                                                            
2025-07-24 07:04:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #22/50                                                            
2025-07-24 07:05:01: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #23/50                                                            
2025-07-24 07:05:43: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #24/50                                                            
2025-07-24 07:06:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #25/50                                                            
2025-07-24 07:07:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #26/50                                                            
2025-07-24 07:07:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #27/50                                                            
2025-07-24 07:08:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #28/50                                                            
2025-07-24 07:08:45: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #28/50                                                            
2025-07-24 07:09:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #29/50                                                            
2025-07-24 07:10:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #30/50                                                            
2025-07-24 07:10:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #30/50                                                            
2025-07-24 07:11:01: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #31/50                                                            
2025-07-24 07:11:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #31/50                                                            
2025-07-24 07:11:48: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #32/50                                                            
2025-07-24 07:12:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #33/50                                                            
2025-07-24 07:12:59: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #34/50                                                            
2025-07-24 07:13:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #35/50                                                            
2025-07-24 07:14:05: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #36/50                                                            
2025-07-24 07:14:38: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 07:15:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #38/50                                                            
2025-07-24 07:15:45: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #39/50                                                            
2025-07-24 07:16:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #40/50                                                            
2025-07-24 07:16:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #41/50                                                            
2025-07-24 07:17:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #41/50                                                            
2025-07-24 07:17:40: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #42/50                                                            
2025-07-24 07:18:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #43/50                                                            
2025-07-24 07:20:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #43/50                                                            
2025-07-24 07:20:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #44/50                                                            
2025-07-24 07:21:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #45/50                                                            
2025-07-24 07:21:58: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #46/50                                                            
2025-07-24 07:22:29: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #47/50                                                            
2025-07-24 07:23:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #48/50                                                            
2025-07-24 07:23:35: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #49/50                                                            
2025-07-24 07:24:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 07:25:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 07:25:41: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, requested 50 jobs, got 50, 57.48 s/job                                               
2025-07-24 07:25:57: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #1/50 start                                                                     
2025-07-24 07:26:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #2/50 start                                                                     
2025-07-24 07:26:33: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #3/50 start                                                                     
2025-07-24 07:26:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #4/50 start                                                                     
2025-07-24 07:27:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #5/50 start                                                                     
2025-07-24 07:27:28: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #6/50 start                                                                     
2025-07-24 07:27:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #7/50 start                                                                     
2025-07-24 07:28:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #8/50 start                                                                     
2025-07-24 07:28:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #9/50 start                                                                     
2025-07-24 07:28:41: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #10/50 start                                                                    
2025-07-24 07:29:00: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #11/50 start                                                                    
2025-07-24 07:29:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #12/50 start                                                                    
2025-07-24 07:29:37: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #13/50 start                                                                    
2025-07-24 07:29:56: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #14/50 start                                                                    
2025-07-24 07:30:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #15/50 start                                                                    
2025-07-24 07:30:32: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #16/50 start                                                                    
2025-07-24 07:30:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #17/50 start                                                                    
2025-07-24 07:31:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #18/50 start                                                                    
2025-07-24 07:31:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #19/50 start                                                                    
2025-07-24 07:31:43: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #20/50 start                                                                    
2025-07-24 07:32:00: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #21/50 start                                                                    
2025-07-24 07:32:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #22/50 start                                                                    
2025-07-24 07:32:36: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #23/50 start                                                                    
2025-07-24 07:32:52: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #24/50 start                                                                    
2025-07-24 07:33:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #25/50 start                                                                    
2025-07-24 07:33:29: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #26/50 start                                                                    
2025-07-24 07:33:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #27/50 start                                                                    
2025-07-24 07:34:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #28/50 start                                                                    
2025-07-24 07:34:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #29/50 start                                                                    
2025-07-24 07:34:40: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #30/50 start                                                                    
2025-07-24 07:34:58: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #31/50 start                                                                    
2025-07-24 07:35:16: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #32/50 start                                                                    
2025-07-24 07:35:35: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #33/50 start                                                                    
2025-07-24 07:35:55: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #34/50 start                                                                    
2025-07-24 07:36:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #35/50 start                                                                    
2025-07-24 07:36:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #36/50 start                                                                    
2025-07-24 07:36:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #37/50 start                                                                    
2025-07-24 07:37:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #38/50 start                                                                    
2025-07-24 07:37:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #39/50 start                                                                    
2025-07-24 07:37:44: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #40/50 start                                                                    
2025-07-24 07:38:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #41/50 start                                                                    
2025-07-24 07:38:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #42/50 start                                                                    
2025-07-24 07:38:39: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #43/50 start                                                                    
2025-07-24 07:38:59: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #44/50 start                                                                    
2025-07-24 07:39:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #45/50 start                                                                    
2025-07-24 07:39:35: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #46/50 start                                                                    
2025-07-24 07:39:55: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #47/50 start                                                                    
2025-07-24 07:40:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #48/50 start                                                                    
2025-07-24 07:40:32: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #49/50 start                                                                    
2025-07-24 07:40:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #50/50 start                                                                    
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:41:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 07:44:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 1 = ∑1/50, started new job                                                   
2025-07-24 07:44:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/5 = ∑6/50, started new job                                         
2025-07-24 07:44:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/5 = ∑6/50, started new job                                         
2025-07-24 07:44:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/5 = ∑6/50, started new job                                         
2025-07-24 07:44:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/5 = ∑6/50, started new job                                         
2025-07-24 07:44:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/6 = ∑7/50, started new job                                         
2025-07-24 07:44:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/11 = ∑12/50, started new job                                       
2025-07-24 07:44:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/11 = ∑12/50, started new job                                       
2025-07-24 07:44:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/11 = ∑12/50, started new job                                       
2025-07-24 07:44:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/15 = ∑16/50, started new job                                       
2025-07-24 07:44:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/15 = ∑16/50, started new job                                       
2025-07-24 07:44:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/11 = ∑12/50, started new job                                       
2025-07-24 07:44:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/15 = ∑16/50, started new job                                       
2025-07-24 07:44:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 1/15 = ∑16/50, started new job                                       
2025-07-24 07:44:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 1/15 = ∑16/50, started new job                                       
2025-07-24 07:44:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 1/15 = ∑16/50, started new job                                       
2025-07-24 07:47:16: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 07:47:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 07:48:06: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/2 = ∑18/50, started new job                                       
2025-07-24 07:48:06: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, started new job                             
2025-07-24 07:48:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:10: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, starting new job                            
2025-07-24 07:48:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:16: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:48:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/2 = ∑18/50, starting new job                                      
2025-07-24 07:50:59: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 18/1 = ∑19/50, started new job                                       
2025-07-24 07:51:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/1 = ∑20/50, started new job                                       
2025-07-24 07:51:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/7 = ∑26/50, started new job                                       
2025-07-24 07:51:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/8 = ∑27/50, started new job                                       
2025-07-24 07:51:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), running/unknown 19/7 = ∑26/50, started new job                                                            
2025-07-24 07:51:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/10 = ∑29/50, started new job                                      
2025-07-24 07:51:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/12 = ∑31/50, started new job                                      
2025-07-24 07:51:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, started new job                                      
2025-07-24 07:51:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, starting new job                                     
2025-07-24 07:51:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 19/13 = ∑32/50, starting new job                                     
2025-07-24 07:51:56: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 29/3 = ∑32/50, starting new job                                      
2025-07-24 07:54:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 32 = ∑32/50, starting new job                                                
2025-07-24 07:54:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, started new job                                                 
2025-07-24 07:54:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, started new job                                                 
2025-07-24 07:54:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, started new job                                                 
2025-07-24 07:55:02: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:03: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:03: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:09: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 35 = ∑35/50, starting new job                                                
2025-07-24 07:55:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 35/1 = ∑36/50, started new job                                       
2025-07-24 07:58:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 36/1 = ∑37/50, started new job                                       
2025-07-24 07:58:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 36/2 = ∑38/50, starting new job                                      
2025-07-24 07:58:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 36/2 = ∑38/50, starting new job                                      
2025-07-24 07:58:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 36/3 = ∑39/50, started new job                                       
2025-07-24 07:58:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 36/3 = ∑39/50, started new job                                       
2025-07-24 07:58:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 36/3/1 = ∑40/50, started new job                             
2025-07-24 07:58:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 36/4/1 = ∑41/50, started new job                             
2025-07-24 07:58:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 40/3 = ∑43/50, started new job                                       
2025-07-24 07:58:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 40/3 = ∑43/50, started new job                                       
2025-07-24 07:58:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 40/5 = ∑45/50, started new job                                       
2025-07-24 07:58:28: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 43/2 = ∑45/50, started new job                                       
2025-07-24 07:58:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 43/5 = ∑48/50, started new job                                       
2025-07-24 07:58:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 43/5 = ∑48/50, started new job                                       
2025-07-24 07:58:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 43/5 = ∑48/50, started new job                                       
2025-07-24 08:01:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 48/2 = ∑50/50, started new job                                       
2025-07-24 08:01:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 48/2 = ∑50/50, started new job                                       
2025-07-24 08:02:20: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:02:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:03:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:03:40: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:04:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:04:35: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:05:02: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:05:28: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:05:54: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:06:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:06:47: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:07:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:07:38: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:08:03: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:08:29: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:08:54: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:09:21: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:09:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:10:13: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:10:39: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:11:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:11:32: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:11:59: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:12:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:12:52: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:13:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:13:44: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:14:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:14:37: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:15:05: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:15:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:15:57: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:16:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:16:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:17:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:17:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:18:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:18:54: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:19:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:19:58: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:20:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:21:02: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:21:33: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:22:05: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:22:38: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:23:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:23:42: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:24:14: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:24:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:25:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:25:50: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:26:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:26:55: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:27:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:27:58: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:28:30: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:29:01: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:29:32: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:30:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:30:36: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:31:08: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:31:40: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:32:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:32:43: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:33:15: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:33:47: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:34:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:34:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:35:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:35:55: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:36:28: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:37:01: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:37:34: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:38:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:38:39: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:39:12: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:39:46: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:40:18: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:40:52: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:41:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:41:55: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:42:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:42:59: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:43:31: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:44:03: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:44:34: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:45:07: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:45:39: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:46:11: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:46:45: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:47:17: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:47:48: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:48:19: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:48:51: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:49:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:49:58: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:50:29: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:51:04: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:51:37: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 50 = ∑50/50, waiting for 50 jobs                                             
2025-07-24 08:52:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:23: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:22: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:24: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:25: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:26: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 08:52:27: BoTorchGenerator, failed: 262 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 29/21 = ∑50/50, job_failed                                           
2025-07-24 09:01:39: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 21 = ∑21/50, waiting for 50 jobs, finished 29 jobs                           
2025-07-24 09:19:34: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #1/50                                                             
2025-07-24 09:20:27: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #2/50                                                             
2025-07-24 09:21:18: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #3/50                                                             
2025-07-24 09:22:09: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #4/50                                                             
2025-07-24 09:23:00: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #5/50                                                             
2025-07-24 09:24:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #6/50                                                             
2025-07-24 09:25:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #7/50                                                             
2025-07-24 09:25:38: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #7/50                                                             
2025-07-24 09:26:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #8/50                                                             
2025-07-24 09:27:20: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #9/50                                                             
2025-07-24 09:28:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 09:28:36: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #10/50                                                            
2025-07-24 09:29:26: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #11/50                                                            
2025-07-24 09:29:50: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #11/50                                                            
2025-07-24 09:30:41: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #12/50                                                            
2025-07-24 09:31:31: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 09:31:55: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #13/50                                                            
2025-07-24 09:32:46: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #14/50                                                            
2025-07-24 09:33:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #15/50                                                            
2025-07-24 09:34:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #16/50                                                            
2025-07-24 09:35:19: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #17/50                                                            
2025-07-24 09:36:08: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #18/50                                                            
2025-07-24 09:36:59: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #19/50                                                            
2025-07-24 09:37:50: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #20/50                                                            
2025-07-24 09:38:41: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #21/50                                                            
2025-07-24 09:39:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #22/50                                                            
2025-07-24 09:40:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #23/50                                                            
2025-07-24 09:41:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #24/50                                                            
2025-07-24 09:42:05: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #25/50                                                            
2025-07-24 09:42:57: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #26/50                                                            
2025-07-24 09:43:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #27/50                                                            
2025-07-24 09:44:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #28/50                                                            
2025-07-24 09:45:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #29/50                                                            
2025-07-24 09:46:19: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #30/50                                                            
2025-07-24 09:47:10: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #31/50                                                            
2025-07-24 09:48:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #32/50                                                            
2025-07-24 09:48:55: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #33/50                                                            
2025-07-24 09:49:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #34/50                                                            
2025-07-24 09:50:38: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #35/50                                                            
2025-07-24 09:51:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #36/50                                                            
2025-07-24 09:52:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 09:52:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 09:53:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #37/50                                                            
2025-07-24 09:54:03: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #38/50                                                            
2025-07-24 09:54:55: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #39/50                                                            
2025-07-24 09:55:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #40/50                                                            
2025-07-24 09:56:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #41/50                                                            
2025-07-24 09:57:31: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #42/50                                                            
2025-07-24 09:58:21: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #43/50                                                            
2025-07-24 09:59:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #44/50                                                            
2025-07-24 10:02:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #45/50                                                            
2025-07-24 10:03:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #46/50                                                            
2025-07-24 10:04:07: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #47/50                                                            
2025-07-24 10:04:59: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #48/50                                                            
2025-07-24 10:05:49: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #49/50                                                            
2025-07-24 10:06:42: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 10:07:53: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, getting new HP set #50/50                                                            
2025-07-24 10:08:43: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, requested 50 jobs, got 50, 79.76 s/job                                               
2025-07-24 10:09:06: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #1/50 start                                                                     
2025-07-24 10:09:34: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #2/50 start                                                                     
2025-07-24 10:10:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #3/50 start                                                                     
2025-07-24 10:10:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #4/50 start                                                                     
2025-07-24 10:10:56: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #5/50 start                                                                     
2025-07-24 10:11:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #6/50 start                                                                     
2025-07-24 10:11:51: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #7/50 start                                                                     
2025-07-24 10:12:19: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #8/50 start                                                                     
2025-07-24 10:12:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #9/50 start                                                                     
2025-07-24 10:13:14: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #10/50 start                                                                    
2025-07-24 10:13:41: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #11/50 start                                                                    
2025-07-24 10:14:10: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #12/50 start                                                                    
2025-07-24 10:14:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #13/50 start                                                                    
2025-07-24 10:15:06: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #14/50 start                                                                    
2025-07-24 10:15:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #15/50 start                                                                    
2025-07-24 10:16:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #16/50 start                                                                    
2025-07-24 10:16:29: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #17/50 start                                                                    
2025-07-24 10:16:56: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #18/50 start                                                                    
2025-07-24 10:17:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #19/50 start                                                                    
2025-07-24 10:17:50: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #20/50 start                                                                    
2025-07-24 10:18:18: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #21/50 start                                                                    
2025-07-24 10:18:45: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #22/50 start                                                                    
2025-07-24 10:19:14: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #23/50 start                                                                    
2025-07-24 10:19:42: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #24/50 start                                                                    
2025-07-24 10:20:09: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #25/50 start                                                                    
2025-07-24 10:20:36: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #26/50 start                                                                    
2025-07-24 10:21:04: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #27/50 start                                                                    
2025-07-24 10:21:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #28/50 start                                                                    
2025-07-24 10:21:59: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #29/50 start                                                                    
2025-07-24 10:22:27: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #30/50 start                                                                    
2025-07-24 10:22:54: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #31/50 start                                                                    
2025-07-24 10:23:22: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #32/50 start                                                                    
2025-07-24 10:23:49: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #33/50 start                                                                    
2025-07-24 10:24:17: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #34/50 start                                                                    
2025-07-24 10:24:45: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #35/50 start                                                                    
2025-07-24 10:25:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #36/50 start                                                                    
2025-07-24 10:25:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #37/50 start                                                                    
2025-07-24 10:26:07: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #38/50 start                                                                    
2025-07-24 10:26:34: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #39/50 start                                                                    
2025-07-24 10:27:02: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #40/50 start                                                                    
2025-07-24 10:27:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #41/50 start                                                                    
2025-07-24 10:27:57: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #42/50 start                                                                    
2025-07-24 10:28:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #43/50 start                                                                    
2025-07-24 10:29:04: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #44/50 start                                                                    
2025-07-24 10:29:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #45/50 start                                                                    
2025-07-24 10:30:00: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #46/50 start                                                                    
2025-07-24 10:30:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #47/50 start                                                                    
2025-07-24 10:30:57: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #48/50 start                                                                    
2025-07-24 10:31:27: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #49/50 start                                                                    
2025-07-24 10:31:54: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, eval #50/50 start                                                                    
2025-07-24 10:32:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:32:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, starting new job                                                                     
2025-07-24 10:37:04: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, unknown 2 = ∑2/50, started new job                                                   
2025-07-24 10:37:06: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 2 = ∑2/50, started new job                                                   
2025-07-24 10:37:07: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 2/3 = ∑5/50, started new job                                         
2025-07-24 10:37:08: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 2/3 = ∑5/50, started new job                                         
2025-07-24 10:37:08: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 2/4 = ∑6/50, started new job                                         
2025-07-24 10:37:10: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 2/4 = ∑6/50, started new job                                         
2025-07-24 10:37:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:13: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:14: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:16: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending 16 = ∑16/50, started new job                                                 
2025-07-24 10:37:16: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:37:17: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, pending/unknown 7/9 = ∑16/50, started new job                                        
2025-07-24 10:42:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:42:42: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:42:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:42:56: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:42:57: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:06: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:08: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:15: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:22: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:22: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:43:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:45:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 16 = ∑16/50, starting new job                                                
2025-07-24 10:46:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 16/1 = ∑17/50, started new job                                       
2025-07-24 10:47:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/1/1 = ∑18/50, started new job                             
2025-07-24 10:47:44: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/2/1 = ∑19/50, started new job                             
2025-07-24 10:47:56: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/3/1 = ∑20/50, started new job                             
2025-07-24 10:47:58: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/5 = ∑21/50, started new job                                       
2025-07-24 10:48:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/5/2 = ∑23/50, started new job                             
2025-07-24 10:48:03: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/7/1 = ∑24/50, started new job                             
2025-07-24 10:48:04: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/7/1 = ∑24/50, started new job                             
2025-07-24 10:48:18: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/8/1 = ∑25/50, started new job                             
2025-07-24 10:48:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/9/1 = ∑26/50, started new job                             
2025-07-24 10:48:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/10/5 = ∑31/50, started new job                            
2025-07-24 10:48:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/10/5 = ∑31/50, started new job                            
2025-07-24 10:48:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/10/5 = ∑31/50, started new job                            
2025-07-24 10:48:29: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/15 = ∑31/50, started new job                                      
2025-07-24 10:48:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/15 = ∑31/50, started new job                                      
2025-07-24 10:50:29: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 16/15/1 = ∑32/50, started new job                            
2025-07-24 10:51:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 16/16 = ∑32/50, starting new job                                     
2025-07-24 10:53:17: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:53:45: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:53:51: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:53:53: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:39: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best , running/pending 17/15 = ∑32/50, starting new job                                                   
2025-07-24 10:54:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:49: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:54:50: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:55:09: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 17/15 = ∑32/50, starting new job                                     
2025-07-24 10:55:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/unknown 32/1 = ∑33/50, started new job                                       
2025-07-24 10:56:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 33/1 = ∑34/50, started new job                                       
2025-07-24 10:57:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/1/1 = ∑35/50, started new job                             
2025-07-24 10:57:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/1/2 = ∑36/50, started new job                             
2025-07-24 10:57:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/3/1 = ∑37/50, started new job                             
2025-07-24 10:58:40: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/4/1 = ∑38/50, started new job                             
2025-07-24 11:00:20: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/5/2 = ∑40/50, started new job                             
2025-07-24 11:00:21: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/5/2 = ∑40/50, started new job                             
2025-07-24 11:00:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/7/1 = ∑41/50, started new job                             
2025-07-24 11:00:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/7/3 = ∑43/50, started new job                             
2025-07-24 11:00:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/7/3 = ∑43/50, started new job                             
2025-07-24 11:00:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/10/5 = ∑48/50, started new job                            
2025-07-24 11:00:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/10/5 = ∑48/50, started new job                            
2025-07-24 11:00:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/10/5 = ∑48/50, started new job                            
2025-07-24 11:00:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/10/5 = ∑48/50, started new job                            
2025-07-24 11:00:32: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 33/10/5 = ∑48/50, started new job                            
2025-07-24 11:00:53: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 33/15 = ∑48/50, starting new job                                     
2025-07-24 11:01:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 33/15 = ∑48/50, starting new job                                     
2025-07-24 11:05:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 43/6/1 = ∑50/50, started new job                             
2025-07-24 11:05:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending/unknown 43/6/1 = ∑50/50, started new job                             
2025-07-24 11:06:58: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:07:34: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:08:10: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:08:45: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:09:22: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:09:58: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:10:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:11:10: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:11:47: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:12:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:13:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:13:36: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:14:13: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:14:49: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:15:24: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:16:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, waiting for 50 jobs                                   
2025-07-24 11:16:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running/pending 43/7 = ∑50/50, new result: 96.9                                      
2025-07-24 11:18:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 50 jobs, finished 1 job                             
2025-07-24 11:18:27: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:19:04: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:19:55: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:20:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:21:05: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:21:39: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:22:12: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:22:45: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:23:16: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:23:48: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:24:20: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:24:52: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:25:23: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:25:54: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:26:26: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:26:57: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:27:30: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:28:02: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:28:35: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, waiting for 49 jobs                                             
2025-07-24 11:29:01: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 49 = ∑49/50, new result: 98.3                                                
2025-07-24 11:30:14: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 49 jobs, finished 1 job                             
2025-07-24 11:30:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:31:07: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:31:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:32:07: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:32:36: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:33:05: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:33:33: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:34:02: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:34:31: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:35:00: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:35:29: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:35:58: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:36:28: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:36:56: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:37:25: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:37:54: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 48 = ∑48/50, waiting for 48 jobs                                             
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:38:37: BoTorchGenerator, failed: 291 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running 10/38 = ∑48/50, job_failed                                           
2025-07-24 11:41:31: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 48 jobs, finished 10 jobs                           
2025-07-24 11:41:51: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:42:19: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:42:47: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:43:15: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:43:45: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:44:14: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:44:42: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:45:10: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:45:39: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:46:07: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, waiting for 38 jobs                                             
2025-07-24 11:46:33: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 38 = ∑38/50, new result: 98.32                                               
2025-07-24 11:47:39: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 37 = ∑37/50, waiting for 38 jobs, finished 1 job                             
2025-07-24 11:47:59: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, running 37 = ∑37/50, waiting for 37 jobs                                             
2025-07-24 11:48:24: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, new result: 98.35                         
2025-07-24 11:48:24: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, new result: 98.47                         
2025-07-24 11:48:40: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:48:41: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:48:41: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:48:41: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:48:41: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:48:42: BoTorchGenerator, failed: 301 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.44, timeout/running/completed 6/30/1 = ∑37/50, job_failed                                
2025-07-24 11:51:41: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 37 jobs, finished 8 jobs                            
2025-07-24 11:52:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:52:31: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:53:01: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:53:29: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:53:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:54:25: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:54:55: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:55:24: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:55:52: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:56:21: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:56:50: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:57:19: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:57:48: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:58:16: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 29 = ∑29/50, waiting for 29 jobs                                             
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:58:58: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:01: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:00: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 11:59:01: BoTorchGenerator, failed: 307 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 15/14 = ∑29/50, job_failed                                           
2025-07-24 12:03:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 29 jobs, finished 15 jobs                           
2025-07-24 12:03:35: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:04:04: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:04:34: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:05:02: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:05:32: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:06:01: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:06:30: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:07:00: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:07:27: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:07:56: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:08:24: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 14 = ∑14/50, waiting for 14 jobs                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:05: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:09:06: BoTorchGenerator, failed: 322 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout/running 9/5 = ∑14/50, job_failed                                             
2025-07-24 12:11:43: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 14 jobs, finished 9 jobs                              
2025-07-24 12:12:01: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:12:29: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:12:58: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:13:26: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:13:54: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:14:21: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:14:48: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:15:16: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:15:42: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:16:09: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:16:37: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:17:03: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:17:32: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:17:59: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:18:27: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, running 5 = ∑5/50, waiting for 5 jobs                                                
2025-07-24 12:19:06: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed                                                        
2025-07-24 12:19:06: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed                                                        
2025-07-24 12:19:06: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed                                                        
2025-07-24 12:19:06: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed                                                        
2025-07-24 12:19:06: BoTorchGenerator, failed: 331 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, timeout 5 = ∑5/50, job_failed                                                        
2025-07-24 12:20:34: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, waiting for 5 jobs, finished 5 jobs                                                  
2025-07-24 12:33:10: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #1/50                                                             
2025-07-24 12:33:51: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #2/50                                                             
2025-07-24 12:34:32: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #3/50                                                             
2025-07-24 12:35:14: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #4/50                                                             
2025-07-24 12:35:57: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #5/50                                                             
2025-07-24 12:36:42: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #6/50                                                             
2025-07-24 12:37:23: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #7/50                                                             
2025-07-24 12:38:06: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #8/50                                                             
2025-07-24 12:38:50: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #9/50                                                             
2025-07-24 12:39:34: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50                                                            
2025-07-24 12:39:53: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #10/50                                                            
2025-07-24 12:40:36: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #11/50                                                            
2025-07-24 12:41:20: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #12/50                                                            
2025-07-24 12:42:02: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #13/50                                                            
2025-07-24 12:42:46: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #14/50                                                            
2025-07-24 12:43:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #15/50                                                            
2025-07-24 12:44:15: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #16/50                                                            
2025-07-24 12:44:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #17/50                                                            
2025-07-24 12:45:39: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50                                                            
2025-07-24 12:45:57: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50                                                            
2025-07-24 12:46:18: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #18/50                                                            
2025-07-24 12:46:58: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #19/50                                                            
2025-07-24 12:47:36: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #20/50                                                            
2025-07-24 12:47:56: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #20/50                                                            
2025-07-24 12:48:37: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #21/50                                                            
2025-07-24 12:49:32: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #22/50                                                            
2025-07-24 12:50:11: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #23/50                                                            
2025-07-24 12:50:52: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #24/50                                                            
2025-07-24 12:51:32: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #25/50                                                            
2025-07-24 12:52:13: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #26/50                                                            
2025-07-24 12:52:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #27/50                                                            
2025-07-24 12:53:45: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #28/50                                                            
2025-07-24 12:54:34: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #29/50                                                            
2025-07-24 12:55:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #30/50                                                            
2025-07-24 12:56:27: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #31/50                                                            
2025-07-24 12:57:22: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #32/50                                                            
2025-07-24 12:57:47: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #32/50                                                            
2025-07-24 12:58:41: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #33/50                                                            
2025-07-24 12:59:42: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #34/50                                                            
2025-07-24 13:00:37: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #35/50                                                            
2025-07-24 13:01:31: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #36/50                                                            
2025-07-24 13:02:25: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #37/50                                                            
2025-07-24 13:03:23: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #38/50                                                            
2025-07-24 13:04:20: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #39/50                                                            
2025-07-24 13:05:15: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #40/50                                                            
2025-07-24 13:06:11: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #41/50                                                            
2025-07-24 13:07:12: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #42/50                                                            
2025-07-24 13:08:09: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #43/50                                                            
2025-07-24 13:09:03: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #44/50                                                            
2025-07-24 13:09:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #45/50                                                            
2025-07-24 13:12:23: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #46/50                                                            
2025-07-24 13:13:19: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #47/50                                                            
2025-07-24 13:14:14: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #48/50                                                            
2025-07-24 13:15:09: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #49/50                                                            
2025-07-24 13:16:04: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, getting new HP set #50/50                                                            
2025-07-24 13:16:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, requested 50 jobs, got 50, 67.16 s/job                                               
2025-07-24 13:17:24: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #1/50 start                                                                     
2025-07-24 13:17:55: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #2/50 start                                                                     
2025-07-24 13:18:26: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #3/50 start                                                                     
2025-07-24 13:18:56: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #4/50 start                                                                     
2025-07-24 13:19:26: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #5/50 start                                                                     
2025-07-24 13:19:57: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #6/50 start                                                                     
2025-07-24 13:20:27: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #7/50 start                                                                     
2025-07-24 13:20:58: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #8/50 start                                                                     
2025-07-24 13:21:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #9/50 start                                                                     
2025-07-24 13:21:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #10/50 start                                                                    
2025-07-24 13:22:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #11/50 start                                                                    
2025-07-24 13:22:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #12/50 start                                                                    
2025-07-24 13:23:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #13/50 start                                                                    
2025-07-24 13:23:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #14/50 start                                                                    
2025-07-24 13:24:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #15/50 start                                                                    
2025-07-24 13:24:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #16/50 start                                                                    
2025-07-24 13:25:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #17/50 start                                                                    
2025-07-24 13:26:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #18/50 start                                                                    
2025-07-24 13:26:29: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #19/50 start                                                                    
2025-07-24 13:26:59: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #20/50 start                                                                    
2025-07-24 13:27:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #21/50 start                                                                    
2025-07-24 13:28:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #22/50 start                                                                    
2025-07-24 13:28:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #23/50 start                                                                    
2025-07-24 13:29:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #24/50 start                                                                    
2025-07-24 13:29:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #25/50 start                                                                    
2025-07-24 13:30:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #26/50 start                                                                    
2025-07-24 13:30:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #27/50 start                                                                    
2025-07-24 13:31:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #28/50 start                                                                    
2025-07-24 13:31:30: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #29/50 start                                                                    
2025-07-24 13:32:00: BoTorchGenerator, failed: 336 ('VAL_ACC: <FLOAT>' not found), best VAL_ACC: 98.47, eval #30/50 start                                                                    

Copy raw data to clipboard

Download »progressbar« as file

Arguments Overview

Copy raw data to clipboard

Download »worker_usage.csv« as file

1753270343.3793383,50,0,0
1753270347.3984902,50,0,0
1753270347.6455984,50,0,0
1753270353.0577917,50,0,0
1753270361.8174677,50,0,0
1753270370.3879108,50,0,0
1753270380.0700917,50,0,0
1753270389.2042003,50,0,0
1753270399.5641398,50,0,0
1753270409.3211598,50,0,0
1753270419.4195647,50,0,0
1753270430.7973082,50,0,0
1753270442.8763926,50,0,0
1753270455.4063902,50,0,0
1753270465.3653016,50,0,0
1753270475.8143132,50,0,0
1753270487.648398,50,0,0
1753270498.7223232,50,0,0
1753270510.5755308,50,0,0
1753270521.9696453,50,0,0
1753270533.3705292,50,0,0
1753270546.4245963,50,0,0
1753270557.0351467,50,0,0
1753270568.335431,50,0,0
1753270578.3440185,50,0,0
1753270589.2304044,50,0,0
1753270597.588638,50,0,0
1753270605.8281686,50,0,0
1753270614.1851132,50,0,0
1753270622.9631305,50,0,0
1753270631.5338743,50,0,0
1753270639.7729137,50,0,0
1753270648.2483382,50,0,0
1753270656.5157897,50,0,0
1753270664.6924388,50,0,0
1753270673.2879674,50,0,0
1753270681.6507561,50,0,0
1753270690.048179,50,0,0
1753270698.4611301,50,0,0
1753270706.668251,50,0,0
1753270716.3264773,50,0,0
1753270724.5947483,50,0,0
1753270732.5785644,50,0,0
1753270740.5263758,50,0,0
1753270748.5783336,50,0,0
1753270757.1201603,50,0,0
1753270765.501985,50,0,0
1753270773.859243,50,0,0
1753270782.18853,50,0,0
1753270790.557819,50,0,0
1753270799.8079278,50,0,0
1753270808.3050346,50,0,0
1753270816.6409447,50,0,0
1753270824.821643,50,0,0
1753270828.934122,50,0,0
1753270833.6291845,50,0,0
1753270839.0010846,50,0,0
1753270844.0712132,50,0,0
1753270849.4407318,50,0,0
1753270853.7894757,50,0,0
1753270858.5290651,50,0,0
1753270862.9919915,50,0,0
1753270867.3098621,50,0,0
1753270871.6839912,50,0,0
1753270876.0883093,50,0,0
1753270880.5772913,50,0,0
1753270885.0603106,50,0,0
1753270889.4209874,50,0,0
1753270895.1673603,50,0,0
1753270899.6328154,50,0,0
1753270904.0311482,50,0,0
1753270908.569887,50,0,0
1753270912.9479773,50,0,0
1753270917.5074098,50,0,0
1753270922.1452913,50,0,0
1753270926.9347806,50,0,0
1753270931.3372743,50,0,0
1753270935.8709545,50,0,0
1753270940.1767957,50,0,0
1753270944.5476794,50,0,0
1753270948.8725884,50,0,0
1753270954.4693778,50,0,0
1753270959.0713885,50,0,0
1753270963.5929074,50,0,0
1753270967.9165144,50,0,0
1753270972.426308,50,0,0
1753270976.8086514,50,0,0
1753270981.0928333,50,0,0
1753270985.5267272,50,0,0
1753270990.0193481,50,0,0
1753270994.4905026,50,0,0
1753270998.7394304,50,0,0
1753271003.1058137,50,0,0
1753271007.5879076,50,0,0
1753271012.0371742,50,0,0
1753271016.41959,50,0,0
1753271020.906116,50,0,0
1753271025.2909477,50,0,0
1753271029.751639,50,0,0
1753271034.41503,50,0,0
1753271038.8063765,50,0,0
1753271043.142643,50,0,0
1753271047.6693444,50,0,0
1753271052.0062804,50,0,0
1753271058.073999,50,0,0
1753271058.085533,50,0,0
1753271058.1354132,50,0,0
1753271058.1589067,50,0,0
1753271058.1614177,50,0,0
1753271058.1821077,50,0,0
1753271058.182977,50,0,0
1753271058.1839051,50,0,0
1753271058.184792,50,0,0
1753271058.1857207,50,0,0
1753271058.186558,50,0,0
1753271058.1873205,50,0,0
1753271058.2417872,50,0,0
1753271058.25224,50,0,0
1753271058.3283541,50,0,0
1753271058.4696484,50,0,0
1753271085.4046912,50,8,16
1753271085.7811444,50,10,20
1753271085.8006694,50,10,20
1753271085.88722,50,10,20
1753271086.369588,50,12,24
1753271086.7108366,50,12,24
1753271087.2063806,50,14,28
1753271087.2786222,50,14,28
1753271087.8068383,50,15,30
1753271088.243432,50,16,32
1753271088.6215756,50,16,32
1753271089.8363364,50,16,32
1753271090.047121,50,16,32
1753271090.3855815,50,16,32
1753271090.6454809,50,16,32
1753271091.023732,50,16,32
1753271126.046495,50,16,32
1753271126.531818,50,16,32
1753271126.7028084,50,16,32
1753271128.0115662,50,16,32
1753271128.7176504,50,16,32
1753271129.0766006,50,16,32
1753271129.1059093,50,16,32
1753271129.577963,50,16,32
1753271130.4006717,50,16,32
1753271130.5874455,50,16,32
1753271131.598182,50,16,32
1753271131.729888,50,16,32
1753271131.8182805,50,16,32
1753271132.150451,50,16,32
1753271160.0945961,50,16,32
1753271165.2217617,50,17,34
1753271166.1844876,50,18,36
1753271167.4820936,50,19,38
1753271167.9203582,50,19,38
1753271173.1752448,50,21,42
1753271174.690668,50,23,46
1753271176.7166505,50,28,56
1753271177.1972907,50,26,52
1753271177.9472678,50,30,60
1753271178.0771334,50,30,60
1753271179.0317187,50,30,60
1753271179.0815551,50,30,60
1753271179.2517173,50,30,60
1753271179.9913533,50,30,60
1753271180.6012347,50,30,60
1753271184.0792313,50,31,62
1753271199.738712,50,32,64
1753271200.815252,50,32,64
1753271203.0914514,50,32,64
1753271204.8292077,50,32,64
1753271215.8609552,50,32,64
1753271216.582681,50,32,64
1753271241.5994396,50,35,70
1753271241.6904697,50,36,72
1753271241.975727,50,36,72
1753271242.3766265,50,37,74
1753271242.4912853,50,37,74
1753271242.5911903,50,37,74
1753271243.3267257,50,37,74
1753271243.498556,50,37,74
1753271243.5566185,50,37,74
1753271244.0432692,50,37,74
1753271244.672491,50,37,74
1753271244.6963246,50,37,74
1753271244.9051394,50,37,74
1753271245.2782276,50,37,74
1753271245.4535804,50,37,74
1753271247.3288157,50,37,74
1753271289.6230693,50,38,76
1753271292.511952,50,40,80
1753271293.1246696,50,40,80
1753271293.20559,50,40,80
1753271300.1481137,50,41,82
1753271307.7383506,50,48,96
1753271307.8906338,50,48,96
1753271307.9993865,50,48,96
1753271308.083712,50,48,96
1753271308.1463187,50,48,96
1753271308.2201648,50,48,96
1753271308.3439958,50,48,96
1753271309.6871166,50,48,96
1753271313.0285022,50,49,98
1753271354.4338713,50,50,100
1753271363.7540562,50,50,100
1753271363.936887,50,50,100
1753271364.2384386,50,50,100
1753271376.3230953,50,50,100
1753271388.3303108,50,50,100
1753271400.723778,50,50,100
1753271412.7375453,50,50,100
1753271428.9970887,50,50,100
1753271441.2084386,50,50,100
1753271453.2398713,50,50,100
1753271465.193747,50,50,100
1753271477.0926173,50,50,100
1753271489.311953,50,50,100
1753271501.5485115,50,50,100
1753271513.5506938,50,49,98
1753271520.7143476,50,49,98
1753271534.9917462,50,49,98
1753271541.356209,50,49,98
1753271553.3823872,50,49,98
1753271565.2306104,50,48,96
1753271572.4705355,50,48,96
1753271585.9453459,50,48,96
1753271592.3460915,50,48,96
1753271604.3046637,50,48,96
1753271616.4605067,50,48,96
1753271628.5352583,50,48,96
1753271640.4978983,50,48,96
1753271652.471195,50,48,96
1753271665.4665136,50,48,96
1753271677.2128024,50,48,96
1753271689.8120518,50,48,96
1753271701.98155,50,48,96
1753271714.3217556,50,48,96
1753271726.330702,50,48,96
1753271738.4574928,50,48,96
1753271751.069098,50,48,96
1753271763.8159676,50,48,96
1753271776.9682326,50,48,96
1753271789.4038546,50,48,96
1753271801.4575498,50,48,96
1753271814.2833323,50,48,96
1753271828.640896,50,48,96
1753271841.0369575,50,48,96
1753271853.5891762,50,48,96
1753271866.0778506,50,48,96
1753271878.8095345,50,48,96
1753271891.3241422,50,48,96
1753271903.8845017,50,48,96
1753271916.2852008,50,48,96
1753271928.1604862,50,48,96
1753271942.9332662,50,48,96
1753271955.2325456,50,48,96
1753271973.49228,50,48,96
1753271985.533444,50,48,96
1753271997.5124047,50,48,96
1753272010.394251,50,48,96
1753272022.6511257,50,48,96
1753272034.5415611,50,48,96
1753272046.5366712,50,48,96
1753272060.1146073,50,48,96
1753272072.8622506,50,48,96
1753272083.4071627,50,47,94
1753272098.6399424,50,47,94
1753272105.1424806,50,47,94
1753272117.2000046,50,47,94
1753272129.5275402,50,47,94
1753272142.2504532,50,47,94
1753272154.3930376,50,47,94
1753272166.3587308,50,47,94
1753272178.8430638,50,47,94
1753272191.2166073,50,47,94
1753272203.3864198,50,47,94
1753272215.8214893,50,47,94
1753272228.1243994,50,47,94
1753272240.4176147,50,47,94
1753272252.7670891,50,47,94
1753272264.9961238,50,47,94
1753272281.4271886,50,47,94
1753272293.600195,50,47,94
1753272311.0275614,50,47,94
1753272323.5281985,50,47,94
1753272338.0166562,50,47,94
1753272350.9499786,50,47,94
1753272363.7527049,50,47,94
1753272376.5005877,50,47,94
1753272390.7581847,50,47,94
1753272403.7407627,50,47,94
1753272418.7855396,50,47,94
1753272431.5581567,50,47,94
1753272444.7712803,50,47,94
1753272457.516063,50,47,94
1753272470.6481354,50,47,94
1753272483.2530227,50,47,94
1753272495.4972396,50,47,94
1753272508.1916003,50,47,94
1753272520.810498,50,47,94
1753272532.9911137,50,47,94
1753272545.1912756,50,47,94
1753272557.6168919,50,47,94
1753272569.7450151,50,47,94
1753272582.2339308,50,47,94
1753272606.1516232,50,47,94
1753272618.0966403,50,47,94
1753272630.4449785,50,47,94
1753272648.9260736,50,47,94
1753272664.693744,50,47,94
1753272677.519164,50,47,94
1753272689.3995962,50,47,94
1753272701.4006903,50,47,94
1753272715.7259963,50,47,94
1753272728.4121056,50,47,94
1753272740.7768624,50,47,94
1753272752.8895123,50,47,94
1753272769.8589728,50,47,94
1753272783.1082785,50,47,94
1753272795.5587273,50,47,94
1753272807.6995595,50,47,94
1753272821.118767,50,47,94
1753272833.2520773,50,47,94
1753272845.3604264,50,47,94
1753272857.3470416,50,47,94
1753272872.0365646,50,46,92
1753272881.293389,50,46,92
1753272894.7506602,50,46,92
1753272901.120146,50,46,92
1753272914.105597,50,46,92
1753272927.1859665,50,46,92
1753272939.6403813,50,45,90
1753272948.0748248,50,45,90
1753272962.6208236,50,45,90
1753272969.335548,50,45,90
1753272982.461741,50,45,90
1753272994.3495708,50,45,90
1753273007.1835546,50,45,90
1753273020.2749019,50,45,90
1753273041.472426,50,45,90
1753273053.8757718,50,45,90
1753273066.0698657,50,45,90
1753273085.3689733,50,45,90
1753273099.9748673,50,45,90
1753273112.42522,50,45,90
1753273124.7609515,50,45,90
1753273136.8469534,50,45,90
1753273149.356604,50,45,90
1753273161.4150908,50,45,90
1753273174.883114,50,44,88
1753273182.1220262,50,44,88
1753273195.599646,50,44,88
1753273202.1199706,50,44,88
1753273214.2603722,50,44,88
1753273233.2993662,50,44,88
1753273245.4586306,50,44,88
1753273258.730558,50,44,88
1753273270.4102879,50,43,86
1753273282.1760194,50,43,86
1753273296.8169234,50,43,86
1753273303.3012214,50,43,86
1753273315.1770618,50,43,86
1753273329.3412638,50,43,86
1753273341.702666,50,43,86
1753273355.0304172,50,43,86
1753273367.4029896,50,43,86
1753273379.5649943,50,43,86
1753273392.2282062,50,43,86
1753273404.5544236,50,43,86
1753273416.920189,50,43,86
1753273429.5068285,50,43,86
1753273441.7826824,50,43,86
1753273453.754291,50,43,86
1753273466.5276804,50,43,86
1753273478.9810493,50,43,86
1753273491.2879422,50,43,86
1753273504.538693,50,43,86
1753273517.3449812,50,43,86
1753273530.262078,50,43,86
1753273542.4555395,50,43,86
1753273554.457274,50,43,86
1753273566.8650672,50,43,86
1753273579.0830019,50,43,86
1753273593.7515428,50,43,86
1753273606.0814955,50,43,86
1753273617.924885,50,42,84
1753273626.0415678,50,42,84
1753273639.7008033,50,42,84
1753273646.5564916,50,42,84
1753273658.77071,50,42,84
1753273671.2906444,50,42,84
1753273683.5481188,50,42,84
1753273696.1214252,50,42,84
1753273708.2740297,50,42,84
1753273721.0890574,50,42,84
1753273733.432581,50,42,84
1753273746.3087032,50,42,84
1753273759.690505,50,42,84
1753273772.4186354,50,42,84
1753273792.6438596,50,42,84
1753273805.6518915,50,42,84
1753273818.4366233,50,42,84
1753273831.4414256,50,42,84
1753273843.8444345,50,42,84
1753273863.5281801,50,42,84
1753273877.151491,50,42,84
1753273899.368925,50,42,84
1753273914.1761694,50,42,84
1753273928.4313507,50,42,84
1753273942.6377068,50,42,84
1753273955.0891936,50,42,84
1753273973.7284257,50,42,84
1753273987.4524255,50,42,84
1753273999.70399,50,42,84
1753274012.1468832,50,42,84
1753274027.8913562,50,42,84
1753274040.4901955,50,42,84
1753274053.690953,50,42,84
1753274066.660495,50,42,84
1753274080.249307,50,42,84
1753274092.7193158,50,42,84
1753274108.1459684,50,42,84
1753274120.7339368,50,42,84
1753274133.5247674,50,41,82
1753274140.2784762,50,41,82
1753274153.975354,50,41,82
1753274161.416185,50,41,82
1753274174.7102017,50,41,82
1753274187.2593687,50,41,82
1753274199.2284374,50,41,82
1753274212.8405056,50,41,82
1753274226.1555877,50,40,80
1753274232.5433128,50,40,80
1753274247.3770251,50,40,80
1753274254.9470344,50,40,80
1753274268.0926387,50,40,80
1753274280.702274,50,40,80
1753274292.7321043,50,40,80
1753274305.4419997,50,40,80
1753274317.8191156,50,40,80
1753274330.8174384,50,40,80
1753274343.2399335,50,40,80
1753274355.8549569,50,40,80
1753274369.3710804,50,40,80
1753274382.4773288,50,40,80
1753274394.7549586,50,40,80
1753274410.6300328,50,39,78
1753274424.5236096,50,39,78
1753274431.7589726,50,39,78
1753274443.886518,50,39,78
1753274455.9849174,50,39,78
1753274468.632077,50,39,78
1753274480.927625,50,39,78
1753274494.1459715,50,38,76
1753274500.7970574,50,38,76
1753274515.5711613,50,38,76
1753274524.1323733,50,38,76
1753274538.1736004,50,38,76
1753274551.1932914,50,38,76
1753274570.1341875,50,38,76
1753274586.0939033,50,38,76
1753274598.5692732,50,38,76
1753274611.4793959,50,38,76
1753274624.748809,50,38,76
1753274638.463005,50,38,76
1753274652.5418205,50,38,76
1753274667.177923,50,38,76
1753274683.8800201,50,38,76
1753274696.8005533,50,37,74
1753274708.590927,50,37,74
1753274725.6836855,50,31,62
1753274732.4422593,50,31,62
1753274748.8815265,50,26,52
1753274770.708774,50,26,52
1753274785.8206928,50,26,52
1753274798.5131066,50,26,52
1753274813.780447,50,18,36
1753274826.6472223,50,18,36
1753274841.8571954,50,13,26
1753274856.2093606,50,13,26
1753274878.0091612,50,9,18
1753274892.5140717,50,9,18
1753274904.7141569,50,9,18
1753274917.860472,50,9,18
1753274930.6184735,50,2,4
1753275015.3779027,50,0,0
1753275032.328869,50,0,0
1753275045.293031,50,0,0
1753275058.3086553,50,0,0
1753275071.8367262,50,0,0
1753275085.680929,50,0,0
1753275109.5422432,50,0,0
1753275130.0985684,50,0,0
1753275144.3927557,50,0,0
1753275157.0782707,50,0,0
1753275170.651624,50,0,0
1753275183.4044743,50,0,0
1753275196.9445274,50,0,0
1753275210.1720908,50,0,0
1753275242.5120778,50,0,0
1753275242.7433505,50,0,0
1753275242.937426,50,0,0
1753275243.069786,50,0,0
1753275243.207011,50,0,0
1753275243.3656657,50,0,0
1753275243.4439073,50,0,0
1753275243.446081,50,0,0
1753275243.4469557,50,0,0
1753275243.4482143,50,0,0
1753275243.4609516,50,0,0
1753275243.487004,50,0,0
1753275243.5961647,50,0,0
1753275243.6313076,50,0,0
1753275244.0260653,50,0,0
1753275244.065783,50,0,0
1753275244.1641634,50,0,0
1753275244.295986,50,0,0
1753275244.3264875,50,0,0
1753275244.4005485,50,0,0
1753275244.401915,50,0,0
1753275244.4496176,50,0,0
1753275244.5634978,50,0,0
1753275244.827876,50,0,0
1753275245.0754437,50,0,0
1753275245.2959526,50,0,0
1753275245.521912,50,0,0
1753275245.6061132,50,0,0
1753275245.86268,50,0,0
1753275245.8638487,50,0,0
1753275246.5038173,50,0,0
1753275249.0884502,50,0,0
1753275308.8921719,50,0,0
1753275312.4122417,50,0,0
1753275312.7650926,50,0,0
1753275313.1477897,50,0,0
1753275313.276908,50,0,0
1753275494.4758751,50,0,0
1753275503.1297061,50,0,0
1753275517.592622,50,0,0
1753275531.3958535,50,0,0
1753275545.3576238,50,0,0
1753275558.4368632,50,0,0
1753275572.3592074,50,0,0
1753275585.6306722,50,0,0
1753275599.1201823,50,0,0
1753275613.3440027,50,0,0
1753275626.679101,50,0,0
1753275641.0716197,50,0,0
1753275654.5221708,50,0,0
1753275668.7888832,50,0,0
1753275682.4873571,50,0,0
1753275706.1373034,50,0,0
1753275720.1187136,50,0,0
1753275733.5483515,50,0,0
1753275746.9321837,50,0,0
1753275765.3931684,50,0,0
1753275778.5020316,50,0,0
1753275791.8719223,50,0,0
1753275807.5124137,50,0,0
1753275822.186024,50,0,0
1753275835.6238456,50,0,0
1753275849.7025316,50,0,0
1753275864.4289079,50,0,0
1753275879.3485408,50,0,0
1753275893.433129,50,0,0
1753275908.0326755,50,0,0
1753275921.8241465,50,0,0
1753275936.423799,50,0,0
1753275951.2995992,50,0,0
1753275972.738082,50,0,0
1753275986.2899373,50,0,0
1753275999.7105587,50,0,0
1753276013.852491,50,0,0
1753276027.8339934,50,0,0
1753276041.9023316,50,0,0
1753276057.1963267,50,0,0
1753276070.775581,50,0,0
1753276085.168639,50,0,0
1753276098.6565225,50,0,0
1753276116.0219514,50,0,0
1753276131.224084,50,0,0
1753276144.831018,50,0,0
1753276158.4528267,50,0,0
1753276176.6921346,50,0,0
1753276191.2490597,50,0,0
1753276204.745323,50,0,0
1753276219.8145626,50,0,0
1753276233.6102703,50,0,0
1753276239.9799795,50,0,0
1753276247.2168605,50,0,0
1753276254.763343,50,0,0
1753276261.9652424,50,0,0
1753276269.2881281,50,0,0
1753276276.3789954,50,0,0
1753276286.4658587,50,0,0
1753276293.6744688,50,0,0
1753276301.4182165,50,0,0
1753276308.72431,50,0,0
1753276315.918666,50,0,0
1753276323.5986521,50,0,0
1753276331.3793128,50,0,0
1753276338.3486156,50,0,0
1753276346.42859,50,0,0
1753276354.3975375,50,0,0
1753276361.455739,50,0,0
1753276372.9656608,50,0,0
1753276385.9136295,50,0,0
1753276393.3240676,50,0,0
1753276400.7958784,50,0,0
1753276408.2702227,50,0,0
1753276415.2776222,50,0,0
1753276422.4899147,50,0,0
1753276429.7642066,50,0,0
1753276437.3750498,50,0,0
1753276445.1606135,50,0,0
1753276452.3519487,50,0,0
1753276459.8379242,50,0,0
1753276467.6545887,50,0,0
1753276479.8787475,50,0,0
1753276489.107298,50,0,0
1753276497.396666,50,0,0
1753276505.1820707,50,0,0
1753276512.426872,50,0,0
1753276520.1848028,50,0,0
1753276527.595489,50,0,0
1753276535.0721655,50,0,0
1753276542.8474889,50,0,0
1753276549.7779562,50,0,0
1753276557.3659518,50,0,0
1753276566.2806633,50,0,0
1753276573.834156,50,0,0
1753276581.7500002,50,0,0
1753276599.8751037,50,0,0
1753276608.9935596,50,0,0
1753276618.5841117,50,0,0
1753276627.2276003,50,0,0
1753276635.8670852,50,0,0
1753276644.2216644,50,0,0
1753276654.315625,50,0,0
1753276654.3166187,50,0,0
1753276654.3272893,50,0,0
1753276654.5373187,50,0,0
1753276654.60935,50,0,0
1753276654.670587,50,0,0
1753276654.685764,50,0,0
1753276654.6931448,50,0,0
1753276654.6958175,50,0,0
1753276654.7006035,50,0,0
1753276654.7387435,50,0,0
1753276654.7398105,50,0,0
1753276654.8274345,50,0,0
1753276654.8286808,50,0,0
1753276655.0371137,50,0,0
1753276655.1231604,50,0,0
1753276728.0022976,50,7,14
1753276728.4750016,50,9,18
1753276729.155229,50,13,26
1753276729.193349,50,13,26
1753276729.2124171,50,13,26
1753276729.376873,50,13,26
1753276730.3816829,50,16,32
1753276730.5943878,50,16,32
1753276730.7862113,50,16,32
1753276731.2754412,50,16,32
1753276731.835429,50,16,32
1753276731.9376285,50,16,32
1753276733.3455436,50,16,32
1753276733.9177613,50,16,32
1753276734.008263,50,16,32
1753276734.363827,50,16,32
1753276816.536987,50,16,32
1753276816.6268537,50,16,32
1753276816.9342496,50,16,32
1753276817.72019,50,16,32
1753276817.7779717,50,16,32
1753276818.4067783,50,16,32
1753276818.6025393,50,16,32
1753276820.2993217,50,16,32
1753276820.3252273,50,16,32
1753276820.3563924,50,16,32
1753276820.8559196,50,16,32
1753276820.9845433,50,16,32
1753276822.0806851,50,16,32
1753276822.1808321,50,16,32
1753276887.5091338,50,17,34
1753276895.089838,50,18,36
1753276895.5106785,50,18,36
1753276897.8679414,50,23,46
1753276898.107465,50,23,46
1753276898.5085275,50,24,48
1753276899.7497165,50,25,50
1753276901.0258374,50,30,60
1753276901.1600738,50,30,60
1753276901.4170444,50,30,60
1753276901.4680672,50,30,60
1753276902.647542,50,30,60
1753276903.0481403,50,30,60
1753276903.2653015,50,30,60
1753276903.339762,50,30,60
1753276904.9053886,50,30,60
1753276909.2150452,50,30,60
1753276967.9468882,50,31,62
1753276968.3641202,50,31,62
1753276975.1848438,50,32,64
1753276980.291033,50,32,64
1753276980.356659,50,32,64
1753276982.3545465,50,32,64
1753276994.8051195,50,33,66
1753276998.2677226,50,33,66
1753276998.654413,50,33,66
1753276999.1280215,50,33,66
1753276999.3837788,50,33,66
1753277000.097733,50,33,66
1753277000.2835402,50,33,66
1753277000.8371682,50,33,66
1753277001.2836676,50,33,66
1753277001.9956186,50,33,66
1753277029.3388658,50,34,68
1753277030.2379727,50,34,68
1753277043.4280214,50,34,68
1753277055.7115307,50,36,72
1753277055.7453532,50,36,72
1753277063.8182023,50,37,74
1753277076.3940053,50,37,74
1753277080.6476157,50,38,76
1753277082.9503832,50,39,78
1753277086.447812,50,46,92
1753277087.0562587,50,46,92
1753277087.1238163,50,46,92
1753277087.3816214,50,46,92
1753277087.9035423,50,46,92
1753277088.1243298,50,46,92
1753277088.2209926,50,46,92
1753277106.454274,50,46,92
1753277115.266274,50,47,94
1753277131.9752479,50,47,94
1753277154.880308,50,48,96
1753277182.958828,50,49,98
1753277193.4808826,50,49,98
1753277204.8825965,50,49,98
1753277227.8191319,50,49,98
1753277247.2781699,50,49,98
1753277247.4745262,50,49,98
1753277247.755137,50,49,98
1753277264.5099678,50,49,98
1753277279.4664488,50,49,98
1753277294.7996888,50,49,98
1753277309.526767,50,49,98
1753277324.0967813,50,49,98
1753277339.1922884,50,49,98
1753277356.246083,50,49,98
1753277370.7251318,50,49,98
1753277385.5129297,50,49,98
1753277408.6757286,50,49,98
1753277424.2976718,50,49,98
1753277439.6048422,50,49,98
1753277454.708302,50,49,98
1753277470.1288018,50,49,98
1753277484.9326968,50,49,98
1753277508.171909,50,49,98
1753277523.1765878,50,49,98
1753277538.1729045,50,49,98
1753277553.2281802,50,48,96
1753277563.9368114,50,48,96
1753277583.5610476,50,48,96
1753277592.3369298,50,48,96
1753277611.304845,50,48,96
1753277626.8243003,50,48,96
1753277641.4590297,50,48,96
1753277657.4819102,50,48,96
1753277672.325435,50,48,96
1753277689.5980096,50,48,96
1753277705.7119493,50,48,96
1753277733.0010371,50,48,96
1753277752.5566566,50,48,96
1753277769.7379758,50,48,96
1753277790.5660486,50,48,96
1753277807.316041,50,48,96
1753277822.8646517,50,48,96
1753277838.52426,50,48,96
1753277854.5592391,50,48,96
1753277869.374159,50,48,96
1753277888.1519926,50,48,96
1753277905.018202,50,48,96
1753277927.528945,50,48,96
1753277944.0733223,50,48,96
1753277959.8178391,50,48,96
1753277974.8375642,50,48,96
1753277993.8179572,50,48,96
1753278010.5380523,50,48,96
1753278039.7581017,50,48,96
1753278057.8500707,50,47,94
1753278067.7821653,50,47,94
1753278091.5201223,50,47,94
1753278109.8868861,50,47,94
1753278127.639602,50,47,94
1753278144.6796668,50,47,94
1753278160.8705654,50,47,94
1753278177.4665227,50,47,94
1753278188.2225168,50,46,92
1753278211.0823283,50,46,92
1753278220.9655929,50,46,92
1753278236.0131252,50,45,90
1753278246.35909,50,45,90
1753278267.272028,50,45,90
1753278279.3360386,50,45,90
1753278297.1470678,50,45,90
1753278313.1822476,50,45,90
1753278355.576493,50,43,86
1753278378.394219,50,43,86
1753278388.2253897,50,43,86
1753278403.615493,50,43,86
1753278414.0236776,50,42,84
1753278452.9966664,50,41,82
1753278462.3806744,50,41,82
1753278478.2630374,50,41,82
1753278478.2686567,50,41,82
1753278492.826329,50,41,82
1753278492.8450441,50,41,82
1753278551.7607732,50,41,82
1753278563.0757852,50,40,80
1753278579.0175004,50,40,80
1753278587.789633,50,40,80
1753278608.0116472,50,40,80
1753278618.2172124,50,40,80
1753278634.4410243,50,40,80
1753278649.8617725,50,40,80
1753278664.756425,50,40,80
1753278679.999453,50,39,78
1753278688.4684525,50,39,78
1753278708.161066,50,39,78
1753278716.6810188,50,39,78
1753278732.0436718,50,39,78
1753278747.389831,50,39,78
1753278762.5240138,50,39,78
1753278776.7613235,50,38,76
1753278796.34398,50,38,76
1753278825.6570916,50,37,74
1753278834.9177415,50,37,74
1753278849.0127861,50,37,74
1753278857.705422,50,37,74
1753278878.1973424,50,37,74
1753278887.020503,50,37,74
1753278901.1483047,50,36,72
1753278910.9851727,50,36,72
1753278931.9254782,50,36,72
1753278941.4547858,50,36,72
1753278956.8726013,50,36,72
1753278972.569039,50,36,72
1753278989.6485248,50,36,72
1753279004.9475713,50,36,72
1753279020.1414185,50,36,72
1753279035.0353885,50,36,72
1753279050.361207,50,36,72
1753279065.3164816,50,36,72
1753279080.181592,50,36,72
1753279096.2841754,50,36,72
1753279113.0627897,50,36,72
1753279129.6018965,50,36,72
1753279144.9492013,50,36,72
1753279159.9975588,50,36,72
1753279175.1049109,50,36,72
1753279190.5275326,50,36,72
1753279205.5899658,50,36,72
1753279222.2827122,50,36,72
1753279237.4298952,50,36,72
1753279252.2653258,50,36,72
1753279271.723827,50,36,72
1753279286.9332075,50,36,72
1753279301.887825,50,36,72
1753279316.955963,50,36,72
1753279332.9193633,50,36,72
1753279348.134955,50,36,72
1753279365.1282434,50,36,72
1753279380.00979,50,36,72
1753279395.427211,50,36,72
1753279410.414869,50,36,72
1753279425.4531717,50,36,72
1753279441.3796682,50,36,72
1753279456.538076,50,36,72
1753279472.414533,50,36,72
1753279493.0340862,50,36,72
1753279509.516658,50,36,72
1753279525.2108538,50,36,72
1753279540.238394,50,36,72
1753279556.1180243,50,36,72
1753279571.2646387,50,36,72
1753279586.168804,50,36,72
1753279601.5033236,50,36,72
1753279616.865362,50,36,72
1753279632.7136188,50,36,72
1753279647.6899595,50,36,72
1753279663.4118998,50,36,72
1753279678.5720074,50,36,72
1753279693.8301902,50,36,72
1753279712.7354944,50,36,72
1753279729.609568,50,36,72
1753279744.822032,50,36,72
1753279760.5841286,50,36,72
1753279777.243373,50,36,72
1753279792.561698,50,36,72
1753279807.6641815,50,36,72
1753279823.4818714,50,36,72
1753279838.4986162,50,36,72
1753279853.392359,50,36,72
1753279868.656606,50,36,72
1753279889.729207,50,36,72
1753279906.1512575,50,36,72
1753279921.985384,50,36,72
1753279937.5356975,50,36,72
1753279952.4726095,50,36,72
1753279967.8512294,50,36,72
1753279983.1071367,50,36,72
1753279998.0244424,50,36,72
1753280017.033873,50,36,72
1753280036.0415034,50,36,72
1753280051.726924,50,36,72
1753280066.8523972,50,36,72
1753280081.8765192,50,36,72
1753280097.6535513,50,36,72
1753280112.5594761,50,36,72
1753280127.5787053,50,36,72
1753280143.7780695,50,36,72
1753280158.6210432,50,36,72
1753280173.6717687,50,36,72
1753280188.6227832,50,36,72
1753280203.8131795,50,36,72
1753280218.8434024,50,36,72
1753280233.7208922,50,36,72
1753280249.1547306,50,36,72
1753280265.092803,50,36,72
1753280280.1833477,50,36,72
1753280295.1325088,50,36,72
1753280310.4844177,50,36,72
1753280326.6192393,50,36,72
1753280343.5333388,50,36,72
1753280359.7128253,50,32,64
1753280373.997713,50,31,62
1753280382.9167135,50,31,62
1753280402.3901389,50,25,50
1753280411.4625034,50,25,50
1753280441.7681558,50,25,50
1753280441.8903584,50,25,50
1753280441.9493206,50,25,50
1753280441.9651816,50,25,50
1753280442.0249083,50,25,50
1753280442.0918663,50,25,50
1753280442.1217368,50,25,50
1753280442.2494936,50,25,50
1753280442.3426802,50,25,50
1753280442.7765794,50,25,50
1753280515.7983663,50,18,36
1753280524.708581,50,18,36
1753280539.8953352,50,15,30
1753280555.3008988,50,15,30
1753280570.3058088,50,15,30
1753280585.740969,50,15,30
1753280600.905083,50,13,26
1753280616.0610113,50,13,26
1753280631.241638,50,13,26
1753280646.1626427,50,13,26
1753280661.3384342,50,12,24
1753280676.5460067,50,12,24
1753280691.7400908,50,8,16
1753280706.817224,50,8,16
1753280722.0877733,50,1,2
1753280737.5962389,50,1,2
1753280753.6749918,50,1,2
1753280769.0910864,50,1,2
1753280784.3712974,50,1,2
1753280799.322503,50,1,2
1753280814.3301218,50,1,2
1753280830.6905427,50,1,2
1753280845.5999837,50,0,0
1753280860.5359957,50,0,0
1753280875.5508327,50,0,0
1753280890.6136146,50,0,0
1753280906.2719083,50,0,0
1753280922.313411,50,0,0
1753280937.5052257,50,0,0
1753280953.3661218,50,0,0
1753280968.514136,50,0,0
1753280983.6437805,50,0,0
1753280999.6206014,50,0,0
1753281014.4282029,50,0,0
1753281046.316765,50,0,0
1753281046.499426,50,0,0
1753281046.7344778,50,0,0
1753281046.7634254,50,0,0
1753281046.7647066,50,0,0
1753281046.7658517,50,0,0
1753281046.8498363,50,0,0
1753281046.9331417,50,0,0
1753281047.1197023,50,0,0
1753281047.3025067,50,0,0
1753281047.3191302,50,0,0
1753281047.3269112,50,0,0
1753281047.399344,50,0,0
1753281047.6442487,50,0,0
1753281047.866429,50,0,0
1753281047.8749456,50,0,0
1753281048.1831214,50,0,0
1753281048.2666872,50,0,0
1753281048.336658,50,0,0
1753281048.4854462,50,0,0
1753281048.7261631,50,0,0
1753281049.0845127,50,0,0
1753281049.1567023,50,0,0
1753281049.7243123,50,0,0
1753281050.3719862,50,0,0
1753281202.1188583,50,0,0
1753281500.7141826,50,0,0
1753281515.3939106,50,0,0
1753281530.9639506,50,0,0
1753281545.823003,50,0,0
1753281559.6432962,50,0,0
1753281573.307716,50,0,0
1753281587.2286894,50,0,0
1753281602.6110103,50,0,0
1753281616.0692093,50,0,0
1753281629.9770567,50,0,0
1753281661.3281407,50,0,0
1753281674.5949242,50,0,0
1753281688.1199675,50,0,0
1753281701.5447834,50,0,0
1753281715.73099,50,0,0
1753281730.023946,50,0,0
1753281743.6489031,50,0,0
1753281756.854055,50,0,0
1753281770.1873343,50,0,0
1753281786.4005008,50,0,0
1753281804.1095705,50,0,0
1753281821.924827,50,0,0
1753281844.2809534,50,0,0
1753281880.059495,50,0,0
1753281894.7703507,50,0,0
1753281909.6458151,50,0,0
1753281926.8983843,50,0,0
1753281942.4912148,50,0,0
1753281959.6540184,50,0,0
1753281975.6423943,50,0,0
1753281992.374151,50,0,0
1753282007.4428709,50,0,0
1753282023.339283,50,0,0
1753282038.3590438,50,0,0
1753282052.205478,50,0,0
1753282069.754681,50,0,0
1753282085.3006394,50,0,0
1753282104.3014014,50,0,0
1753282204.7739124,50,0,0
1753282219.9484978,50,0,0
1753282237.5341082,50,0,0
1753282251.661511,50,0,0
1753282266.5028505,50,0,0
1753282283.6820765,50,0,0
1753282299.600281,50,0,0
1753282316.1035898,50,0,0
1753282330.4131927,50,0,0
1753282348.6786177,50,0,0
1753282364.10633,50,0,0
1753282379.3272858,50,0,0
1753282394.0596495,50,0,0
1753282401.8513901,50,0,0
1753282410.0586717,50,0,0
1753282417.5045943,50,0,0
1753282427.5482864,50,0,0
1753282435.065174,50,0,0
1753282442.903735,50,0,0
1753282450.5063186,50,0,0
1753282458.839398,50,0,0
1753282467.9322777,50,0,0
1753282475.2163968,50,0,0
1753282484.2651253,50,0,0
1753282495.459559,50,0,0
1753282504.6682212,50,0,0
1753282513.6041703,50,0,0
1753282524.8044915,50,0,0
1753282532.6568432,50,0,0
1753282540.5801456,50,0,0
1753282637.7672427,50,0,0
1753282645.659429,50,0,0
1753282654.4921217,50,0,0
1753282662.1303146,50,0,0
1753282670.0716853,50,0,0
1753282678.3140824,50,0,0
1753282686.4107845,50,0,0
1753282693.518235,50,0,0
1753282701.0909145,50,0,0
1753282709.0161622,50,0,0
1753282716.143282,50,0,0
1753282723.0646715,50,0,0
1753282730.554452,50,0,0
1753282738.5051618,50,0,0
1753282746.5357115,50,0,0
1753282754.4660742,50,0,0
1753282762.291646,50,0,0
1753282770.1813338,50,0,0
1753282777.0355737,50,0,0
1753282784.0981228,50,0,0
1753282791.3098452,50,0,0
1753282798.4719875,50,0,0
1753282806.4066942,50,0,0
1753282814.2048547,50,0,0
1753282823.4946375,50,0,0
1753282831.3517244,50,0,0
1753282838.6152048,50,0,0
1753282845.6948152,50,0,0
1753282852.7745879,50,0,0
1753282859.790651,50,0,0
1753282868.2508373,50,0,0
1753282877.737528,50,0,0
1753282887.752436,50,0,0
1753282896.7165573,50,0,0
1753282896.7971723,50,0,0
1753282896.8221934,50,0,0
1753282896.838416,50,0,0
1753282896.8944714,50,0,0
1753282896.90007,50,0,0
1753282896.9069746,50,0,0
1753282896.981043,50,0,0
1753282896.9967997,50,0,0
1753282897.054836,50,0,0
1753282897.076361,50,0,0
1753282897.179779,50,0,0
1753282897.180419,50,0,0
1753282897.2396932,50,0,0
1753282897.3169997,50,0,0
1753282897.4878085,50,0,0
1753282967.5203586,50,5,10
1753282968.2019324,50,8,16
1753282968.2289367,50,8,16
1753282968.589496,50,10,20
1753282969.008309,50,11,22
1753282969.2287698,50,11,22
1753282969.4958317,50,13,26
1753282970.141198,50,16,32
1753282970.4480376,50,16,32
1753282971.073517,50,16,32
1753282971.155581,50,16,32
1753282971.226795,50,16,32
1753282971.896192,50,16,32
1753282971.9713297,50,16,32
1753282972.1710813,50,16,32
1753282972.6480253,50,16,32
1753283033.878173,50,16,32
1753283040.2896712,50,16,32
1753283043.553436,50,16,32
1753283056.0990229,50,17,34
1753283057.4172297,50,18,36
1753283057.8301597,50,18,36
1753283059.3171942,50,18,36
1753283061.1476517,50,18,36
1753283061.9572442,50,18,36
1753283062.893125,50,18,36
1753283063.1761096,50,18,36
1753283063.5126805,50,18,36
1753283063.8286626,50,18,36
1753283064.1961472,50,18,36
1753283064.2536957,50,18,36
1753283064.5461252,50,18,36
1753283079.64917,50,19,38
1753283125.74905,50,21,42
1753283125.8595016,50,21,42
1753283127.9044855,50,21,42
1753283128.34874,50,21,42
1753283130.7116628,50,22,44
1753283134.3282232,50,23,46
1753283134.6270328,50,23,46
1753283135.1177876,50,23,46
1753283140.5650449,50,30,60
1753283140.58997,50,30,60
1753283140.6271625,50,30,60
1753283140.6380537,50,30,60
1753283140.831878,50,30,60
1753283140.9171414,50,30,60
1753283141.0564358,50,30,60
1753283144.2528672,50,30,60
1753283186.104574,50,32,64
1753283186.30276,50,32,64
1753283186.4194593,50,32,64
1753283188.3675864,50,32,64
1753283199.1879349,50,32,64
1753283207.3351288,50,34,68
1753283208.0665708,50,34,68
1753283209.8881595,50,34,68
1753283218.3708713,50,35,70
1753283222.996138,50,35,70
1753283223.6003308,50,35,70
1753283223.6788118,50,35,70
1753283223.8125143,50,35,70
1753283224.233736,50,35,70
1753283224.4951425,50,35,70
1753283224.5281723,50,35,70
1753283261.3976502,50,37,74
1753283262.5545492,50,37,74
1753283264.5679898,50,37,74
1753283265.6593542,50,37,74
1753283274.4578235,50,38,76
1753283283.0276074,50,39,78
1753283285.9745858,50,39,78
1753283286.0688925,50,39,78
1753283303.0045977,50,46,92
1753283303.0305495,50,46,92
1753283303.0647163,50,46,92
1753283303.3705394,50,46,92
1753283303.8008187,50,46,92
1753283304.0944664,50,46,92
1753283304.2163227,50,46,92
1753283335.5504873,50,48,96
1753283335.629789,50,48,96
1753283349.8181398,50,50,100
1753283350.218298,50,50,100
1753283387.6963124,50,50,100
1753283387.8200557,50,50,100
1753283388.020245,50,50,100
1753283402.3988657,50,50,100
1753283417.6754794,50,50,100
1753283432.0955195,50,50,100
1753283446.4442654,50,50,100
1753283460.4165645,50,50,100
1753283474.5705678,50,50,100
1753283488.7459009,50,50,100
1753283503.0568674,50,50,100
1753283517.126539,50,50,100
1753283532.1273482,50,50,100
1753283546.2615342,50,50,100
1753283561.3940544,50,50,100
1753283575.8195517,50,50,100
1753283590.2669811,50,50,100
1753283604.6321816,50,50,100
1753283619.7006652,50,50,100
1753283634.1085813,50,50,100
1753283648.351024,50,50,100
1753283662.425956,50,50,100
1753283676.6325939,50,50,100
1753283691.7658448,50,50,100
1753283707.4970608,50,50,100
1753283724.190133,50,50,100
1753283738.334852,50,50,100
1753283752.874079,50,50,100
1753283767.6063757,50,50,100
1753283781.6130197,50,50,100
1753283796.327345,50,50,100
1753283810.4889681,50,50,100
1753283824.620772,50,50,100
1753283840.7979321,50,50,100
1753283857.1834521,50,50,100
1753283871.1058698,50,50,100
1753283886.869336,50,50,100
1753283901.6383424,50,50,100
1753283916.9834547,50,50,100
1753283935.0725183,50,50,100
1753283949.5756648,50,50,100
1753283963.750274,50,50,100
1753283978.8656795,50,50,100
1753283994.142056,50,50,100
1753284008.9844694,50,50,100
1753284024.200585,50,50,100
1753284038.7705197,50,50,100
1753284053.4776433,50,50,100
1753284068.5043097,50,50,100
1753284083.6548247,50,50,100
1753284099.1348562,50,50,100
1753284114.4773378,50,50,100
1753284129.3238552,50,50,100
1753284144.219676,50,50,100
1753284158.772936,50,50,100
1753284173.65718,50,50,100
1753284189.4803813,50,50,100
1753284204.324291,50,50,100
1753284220.2310224,50,50,100
1753284234.1370013,50,49,98
1753284244.5123482,50,49,98
1753284262.292728,50,49,98
1753284270.8397834,50,49,98
1753284285.6792095,50,49,98
1753284301.5022988,50,49,98
1753284315.5296378,50,49,98
1753284329.8777137,50,49,98
1753284343.800124,50,49,98
1753284359.0510297,50,49,98
1753284374.540584,50,48,96
1753284383.666498,50,47,94
1753284402.0025313,50,47,94
1753284410.0295599,50,47,94
1753284423.8998518,50,47,94
1753284432.5373569,50,47,94
1753284450.790899,50,47,94
1753284458.8374047,50,47,94
1753284473.085779,50,47,94
1753284487.3195965,50,47,94
1753284502.3821514,50,47,94
1753284517.269481,50,47,94
1753284532.1249933,50,47,94
1753284546.9887173,50,47,94
1753284561.643307,50,47,94
1753284575.813322,50,47,94
1753284590.6800451,50,47,94
1753284604.0583148,50,46,92
1753284612.6001334,50,46,92
1753284639.785833,50,44,88
1753284649.3502421,50,43,86
1753284664.6217015,50,43,86
1753284664.6306102,50,43,86
1753284664.6905415,50,43,86
1753284681.269493,50,43,86
1753284681.2858279,50,43,86
1753284682.3563347,50,43,86
1753284722.9010746,50,42,84
1753284732.2447958,50,42,84
1753284745.8450356,50,41,82
1753284745.8592074,50,41,82
1753284758.6301012,50,41,82
1753284758.6305244,50,41,82
1753284794.8943074,50,40,80
1753284802.8453057,50,40,80
1753284816.8689349,50,39,78
1753284816.873833,50,39,78
1753284830.2507033,50,39,78
1753284830.2670908,50,39,78
1753284855.598347,50,39,78
1753284864.854192,50,39,78
1753284882.4931,50,39,78
1753284900.3003063,50,39,78
1753284917.6393807,50,38,76
1753284929.5282722,50,37,74
1753284951.489452,50,37,74
1753284960.1503913,50,37,74
1753284974.0548823,50,37,74
1753284982.6726856,50,37,74
1753285003.120001,50,37,74
1753285012.1400728,50,37,74
1753285027.3960578,50,37,74
1753285042.8575141,50,37,74
1753285058.3709178,50,37,74
1753285074.5375867,50,37,74
1753285089.9481025,50,37,74
1753285105.607587,50,37,74
1753285123.5269074,50,37,74
1753285138.050783,50,36,72
1753285146.7639253,50,36,72
1753285166.1101563,50,36,72
1753285175.3993976,50,36,72
1753285190.0292454,50,36,72
1753285204.6934571,50,36,72
1753285220.6184368,50,36,72
1753285236.7272813,50,35,70
1753285247.3271708,50,35,70
1753285269.8439517,50,35,70
1753285280.0806923,50,35,70
1753285296.6825225,50,35,70
1753285314.01064,50,35,70
1753285329.325719,50,35,70
1753285344.810296,50,35,70
1753285360.2591598,50,35,70
1753285378.0839097,50,35,70
1753285393.3566496,50,35,70
1753285408.494282,50,35,70
1753285425.9194424,50,35,70
1753285443.4285884,50,35,70
1753285460.8555498,50,35,70
1753285479.494254,50,35,70
1753285498.6299837,50,35,70
1753285513.2499316,50,35,70
1753285529.612648,50,35,70
1753285545.7500663,50,34,68
1753285555.3174963,50,34,68
1753285574.2537594,50,34,68
1753285585.0453537,50,34,68
1753285601.3407273,50,34,68
1753285616.1857438,50,33,66
1753285625.1421783,50,33,66
1753285645.142401,50,33,66
1753285653.9819968,50,33,66
1753285669.3676016,50,33,66
1753285684.89648,50,33,66
1753285700.195723,50,33,66
1753285716.6847284,50,33,66
1753285731.8874998,50,33,66
1753285748.6222093,50,33,66
1753285765.1174953,50,33,66
1753285780.2822037,50,33,66
1753285796.4561272,50,33,66
1753285812.5708408,50,33,66
1753285828.5154862,50,33,66
1753285844.8905776,50,33,66
1753285861.834442,50,33,66
1753285877.4438093,50,33,66
1753285893.4755738,50,33,66
1753285910.131612,50,33,66
1753285925.3983788,50,33,66
1753285933.6336527,50,32,64
1753285952.1187286,50,32,64
1753285961.1040154,50,32,64
1753285975.7833273,50,32,64
1753285991.3457804,50,32,64
1753286005.732162,50,32,64
1753286021.1208167,50,32,64
1753286036.574694,50,32,64
1753286051.6251104,50,32,64
1753286066.3538017,50,32,64
1753286081.0801585,50,32,64
1753286096.1930468,50,32,64
1753286111.241785,50,32,64
1753286126.4781578,50,32,64
1753286142.3701448,50,32,64
1753286157.8413358,50,32,64
1753286173.4215431,50,32,64
1753286188.601521,50,32,64
1753286203.6340802,50,32,64
1753286219.026577,50,32,64
1753286233.5674195,50,32,64
1753286248.8749993,50,32,64
1753286264.2650552,50,32,64
1753286280.3986812,50,32,64
1753286296.7609615,50,32,64
1753286311.3067772,50,32,64
1753286327.8141925,50,32,64
1753286342.8417077,50,31,62
1753286352.0450053,50,31,62
1753286371.4200528,50,31,62
1753286380.0675247,50,31,62
1753286394.3672009,50,31,62
1753286408.995456,50,31,62
1753286423.3789034,50,31,62
1753286438.8105183,50,31,62
1753286453.2594576,50,31,62
1753286468.0835059,50,31,62
1753286483.1478796,50,31,62
1753286499.173045,50,31,62
1753286514.8214958,50,31,62
1753286529.9802063,50,31,62
1753286545.096483,50,31,62
1753286559.8809905,50,31,62
1753286574.8456655,50,31,62
1753286590.249471,50,28,56
1753286606.0499868,50,28,56
1753286622.760962,50,25,50
1753286637.569114,50,25,50
1753286652.1760461,50,25,50
1753286667.561109,50,25,50
1753286681.1333034,50,23,46
1753286689.5989597,50,23,46
1753286708.8427887,50,23,46
1753286717.9042513,50,23,46
1753286732.6261985,50,23,46
1753286748.2726402,50,22,44
1753286762.6798477,50,22,44
1753286777.92482,50,20,40
1753286792.5173876,50,20,40
1753286808.0152762,50,14,28
1753286822.8484704,50,14,28
1753286838.0585885,50,12,24
1753286854.395636,50,10,20
1753286869.1452584,50,10,20
1753286887.5146508,50,9,18
1753286902.518381,50,9,18
1753286917.616617,50,7,14
1753286932.805568,50,7,14
1753286948.0413399,50,1,2
1753286963.345096,50,1,2
1753286980.440535,50,0,0
1753286996.4234629,50,0,0
1753287019.4865863,50,0,0
1753287035.6126916,50,0,0
1753287050.7310808,50,0,0
1753287065.4665399,50,0,0
1753287080.3048441,50,0,0
1753287111.286214,50,0,0
1753287111.5339222,50,0,0
1753287111.5871572,50,0,0
1753287111.7050688,50,0,0
1753287112.0351148,50,0,0
1753287112.06343,50,0,0
1753287112.109637,50,0,0
1753287112.252768,50,0,0
1753287112.2776115,50,0,0
1753287112.3153775,50,0,0
1753287112.379952,50,0,0
1753287112.4520617,50,0,0
1753287112.4540396,50,0,0
1753287112.4779134,50,0,0
1753287112.5098047,50,0,0
1753287112.7439067,50,0,0
1753287112.7580938,50,0,0
1753287112.8183677,50,0,0
1753287113.066418,50,0,0
1753287113.0795267,50,0,0
1753287113.0866792,50,0,0
1753287113.1303513,50,0,0
1753287113.2878792,50,0,0
1753287113.3698342,50,0,0
1753287113.4217403,50,0,0
1753287113.7094576,50,0,0
1753287113.7458544,50,0,0
1753287113.8578224,50,0,0
1753287114.1678762,50,0,0
1753287118.0340075,50,0,0
1753287331.0609787,50,0,0
1753288191.2999554,50,0,0
1753288215.0503414,50,0,0
1753288238.5318353,50,0,0
1753288262.5309014,50,0,0
1753288286.3220272,50,0,0
1753288309.197527,50,0,0
1753288332.668608,50,0,0
1753288356.1078315,50,0,0
1753288380.3310523,50,0,0
1753288403.634015,50,0,0
1753288428.1033401,50,0,0
1753288451.3719165,50,0,0
1753288474.4970338,50,0,0
1753288498.29319,50,0,0
1753288521.4164677,50,0,0
1753288544.9668977,50,0,0
1753288567.7864685,50,0,0
1753288593.8219965,50,0,0
1753288617.3059356,50,0,0
1753288640.3848665,50,0,0
1753288665.6744192,50,0,0
1753288693.094907,50,0,0
1753288718.2744136,50,0,0
1753288745.581357,50,0,0
1753288771.8980882,50,0,0
1753288798.6691344,50,0,0
1753288825.9380977,50,0,0
1753288849.529064,50,0,0
1753288872.6041286,50,0,0
1753288895.5715783,50,0,0
1753288919.0949855,50,0,0
1753288943.5659091,50,0,0
1753288966.9028268,50,0,0
1753288989.8283827,50,0,0
1753289013.3377109,50,0,0
1753289037.325563,50,0,0
1753289060.1175175,50,0,0
1753289083.391928,50,0,0
1753289106.214439,50,0,0
1753289129.1507225,50,0,0
1753289152.8273346,50,0,0
1753289176.1630487,50,0,0
1753289199.8126788,50,0,0
1753289223.2567272,50,0,0
1753289246.4102538,50,0,0
1753289269.9341335,50,0,0
1753289292.8914175,50,0,0
1753289317.1863275,50,0,0
1753289340.607875,50,0,0
1753289364.1997645,50,0,0
1753289387.6507204,50,0,0
1753289398.3782823,50,0,0
1753289410.7812507,50,0,0
1753289423.5420845,50,0,0
1753289436.4360545,50,0,0
1753289449.535648,50,0,0
1753289462.0424352,50,0,0
1753289474.9754493,50,0,0
1753289487.3017032,50,0,0
1753289499.7779,50,0,0
1753289512.84182,50,0,0
1753289525.3533678,50,0,0
1753289538.0721269,50,0,0
1753289550.9132965,50,0,0
1753289563.4606266,50,0,0
1753289576.2418528,50,0,0
1753289588.794849,50,0,0
1753289601.5456395,50,0,0
1753289614.2831147,50,0,0
1753289627.1012406,50,0,0
1753289640.043889,50,0,0
1753289652.9410949,50,0,0
1753289665.8597088,50,0,0
1753289678.5258853,50,0,0
1753289691.1209126,50,0,0
1753289703.8091924,50,0,0
1753289716.3573322,50,0,0
1753289729.830314,50,0,0
1753289742.3974106,50,0,0
1753289755.180494,50,0,0
1753289767.811309,50,0,0
1753289780.3866115,50,0,0
1753289793.0070813,50,0,0
1753289805.4928663,50,0,0
1753289818.2591035,50,0,0
1753289830.6380439,50,0,0
1753289843.30903,50,0,0
1753289855.8134184,50,0,0
1753289868.452871,50,0,0
1753289881.0167263,50,0,0
1753289893.7901495,50,0,0
1753289906.2957573,50,0,0
1753289918.8858304,50,0,0
1753289931.855581,50,0,0
1753289944.504207,50,0,0
1753289957.1024473,50,0,0
1753289969.909887,50,0,0
1753289982.5099533,50,0,0
1753289995.081247,50,0,0
1753290008.484921,50,0,0
1753290021.3275597,50,0,0
1753290035.5950513,50,0,0
1753290035.735984,50,0,0
1753290035.7408307,50,0,0
1753290035.7828736,50,0,0
1753290035.8059099,50,0,0
1753290035.953672,50,0,0
1753290035.9859912,50,0,0
1753290035.9868734,50,0,0
1753290035.9875813,50,0,0
1753290035.9882188,50,0,0
1753290036.1009822,50,0,0
1753290036.219772,50,0,0
1753290036.220575,50,0,0
1753290036.2214403,50,0,0
1753290036.734642,50,0,0
1753290037.1927066,50,0,0
1753290159.8405032,50,3,6
1753290160.1729236,50,4,8
1753290161.137116,50,5,10
1753290161.671191,50,6,12
1753290163.4898756,50,15,30
1753290164.3340757,50,15,30
1753290165.2961032,50,16,32
1753290165.671118,50,16,32
1753290166.0067778,50,16,32
1753290166.1768208,50,16,32
1753290166.3251724,50,16,32
1753290166.8016386,50,16,32
1753290166.9739175,50,16,32
1753290167.0321574,50,16,32
1753290167.901212,50,16,32
1753290168.022267,50,16,32
1753290302.1638532,50,15,30
1753290303.6968174,50,15,30
1753290322.6123352,50,15,30
1753290325.3873024,50,15,30
1753290327.2812903,50,15,30
1753290328.919422,50,15,30
1753290329.6526499,50,15,30
1753290333.6595442,50,15,30
1753290333.7018013,50,15,30
1753290334.052698,50,15,30
1753290334.4452312,50,15,30
1753290335.2228582,50,15,30
1753290335.437427,50,15,30
1753290335.9290645,50,15,30
1753290382.2908137,50,15,30
1753290400.780024,50,15,30
1753290416.6800845,50,17,34
1753290418.2525945,50,17,34
1753290448.07561,50,18,36
1753290453.123631,50,21,42
1753290453.9749293,50,21,42
1753290454.7756362,50,22,44
1753290456.617798,50,22,44
1753290459.661328,50,23,46
1753290462.4826467,50,29,58
1753290462.7060664,50,29,58
1753290463.0712078,50,29,58
1753290463.953326,50,29,58
1753290464.0055306,50,29,58
1753290464.8823543,50,29,58
1753290506.6039102,50,30,60
1753290525.2737122,50,30,60
1753290557.3450484,50,30,60
1753290604.35348,50,28,56
1753290606.049827,50,28,56
1753290617.0239718,50,28,56
1753290618.1265874,50,28,56
1753290630.368492,50,28,56
1753290630.7809668,50,28,56
1753290631.7493393,50,28,56
1753290631.9406502,50,28,56
1753290632.2619934,50,28,56
1753290635.7615204,50,28,56
1753290637.4122798,50,28,56
1753290648.44155,50,28,56
1753290666.5732324,50,28,56
1753290683.5655158,50,28,56
1753290694.3627381,50,29,58
1753290735.7675624,50,26,52
1753290745.2003863,50,28,56
1753290745.3692212,50,28,56
1753290759.5274014,50,30,60
1753290760.0508783,50,30,60
1753290769.545215,50,31,62
1753290772.3626895,50,34,68
1753290774.8514493,50,37,74
1753290775.1151872,50,37,74
1753290775.7168484,50,37,74
1753290776.809966,50,38,76
1753290777.8526616,50,38,76
1753290778.3983562,50,38,76
1753290807.58631,50,39,78
1753290818.7002053,50,40,80
1753290877.5056045,50,40,80
1753290893.9642572,50,40,80
1753290894.6250334,50,40,80
1753290996.8902304,50,39,78
1753290998.0044901,50,39,78
1753291054.998815,50,39,78
1753291055.0963612,50,39,78
1753291055.1632178,50,39,78
1753291055.1907027,50,39,78
1753291055.2216346,50,39,78
1753291055.2290337,50,39,78
1753291055.453404,50,39,78
1753291055.471793,50,39,78
1753291056.009313,50,39,78
1753291056.9497328,50,39,78
1753291057.023076,50,39,78
1753291148.1479313,50,39,78
1753291159.2786798,50,39,78
1753291160.396862,50,39,78
1753291161.0285413,50,39,78
1753291161.2856336,50,39,78
1753291162.4982126,50,39,78
1753291162.5157623,50,39,78
1753291162.729517,50,39,78
1753291162.7396445,50,39,78
1753291162.7704914,50,39,78
1753291162.7745237,50,39,78
1753291302.7486944,50,37,74
1753291316.8105843,50,37,74
1753291316.8249683,50,37,74
1753291337.3335552,50,37,74
1753291337.3689358,50,37,74
1753291383.1599264,50,37,74
1753291397.7694848,50,37,74
1753291397.9409127,50,37,74
1753291398.2348142,50,37,74
1753291416.940826,50,36,72
1753291430.3374407,50,36,72
1753291475.390244,50,34,68
1753291488.6755357,50,34,68
1753291507.6472294,50,34,68
1753291507.6581116,50,34,68
1753291528.041665,50,34,68
1753291528.097475,50,34,68
1753291575.7778692,50,34,68
1753291590.374331,50,34,68
1753291610.9735873,50,34,68
1753291632.0664835,50,34,68
1753291653.5814316,50,34,68
1753291672.4818892,50,33,66
1753291685.7781103,50,33,66
1753291717.3512065,50,33,66
1753291731.005399,50,33,66
1753291751.2349772,50,33,66
1753291771.988954,50,33,66
1753291792.5349023,50,33,66
1753291813.5286114,50,33,66
1753291834.4870553,50,33,66
1753291855.3003078,50,33,66
1753291874.158464,50,32,64
1753291887.6710966,50,31,62
1753291918.185641,50,31,62
1753291931.4986913,50,31,62
1753291950.2745056,50,31,62
1753291964.0708694,50,31,62
1753291996.6468468,50,31,62
1753292011.2074096,50,31,62
1753292032.2156513,50,31,62
1753292052.6558664,50,31,62
1753292073.5238078,50,31,62
1753292094.5198638,50,31,62
1753292116.2990623,50,31,62
1753292135.1454449,50,30,60
1753292148.7737978,50,30,60
1753292180.2414734,50,30,60
1753292193.92525,50,30,60
1753292215.8812437,50,30,60
1753292236.700973,50,30,60
1753292258.1585782,50,30,60
1753292278.7044587,50,30,60
1753292299.667705,50,30,60
1753292320.961933,50,30,60
1753292342.0277267,50,30,60
1753292362.9866602,50,30,60
1753292383.8914962,50,30,60
1753292404.7402751,50,30,60
1753292425.9453235,50,30,60
1753292447.0078611,50,30,60
1753292467.7439034,50,30,60
1753292488.3992853,50,30,60
1753292509.2790315,50,30,60
1753292529.8671594,50,30,60
1753292551.0310996,50,30,60
1753292572.0646844,50,30,60
1753292592.9574943,50,30,60
1753292614.1895483,50,30,60
1753292636.3520014,50,30,60
1753292657.1945994,50,30,60
1753292678.5889735,50,30,60
1753292699.7382374,50,30,60
1753292718.6167476,50,29,58
1753292731.9784296,50,29,58
1753292763.3200784,50,29,58
1753292777.0017483,50,29,58
1753292797.9829686,50,29,58
1753292818.8569145,50,29,58
1753292839.8531134,50,29,58
1753292860.6720705,50,29,58
1753292882.8456194,50,29,58
1753292903.844204,50,29,58
1753292924.4771461,50,29,58
1753292943.2668617,50,29,58
1753292956.5601358,50,28,56
1753292987.7646272,50,28,56
1753293001.326307,50,28,56
1753293021.9823778,50,28,56
1753293040.569154,50,27,54
1753293053.7631395,50,27,54
1753293084.4044907,50,27,54
1753293097.9022026,50,27,54
1753293117.5471199,50,26,52
1753293132.9673822,50,26,52
1753293164.0043457,50,26,52
1753293177.3916225,50,26,52
1753293198.6044838,50,25,50
1753293219.2150548,50,25,50
1753293232.6496398,50,24,48
1753293277.6321294,50,22,44
1753293291.1628728,50,22,44
1753293310.2303152,50,22,44
1753293310.251557,50,22,44
1753293310.3098269,50,22,44
1753293339.4843273,50,22,44
1753293339.4848154,50,22,44
1753293339.7092922,50,22,44
1753293421.8027391,50,20,40
1753293435.5228453,50,20,40
1753293454.7350025,50,20,40
1753293454.7357273,50,20,40
1753293474.9146917,50,20,40
1753293474.940565,50,20,40
1753293522.2932193,50,20,40
1753293535.9440615,50,20,40
1753293557.6286192,50,20,40
1753293578.7811933,50,20,40
1753293600.4797873,50,20,40
1753293622.144455,50,20,40
1753293643.2073734,50,20,40
1753293664.431861,50,20,40
1753293685.6047587,50,20,40
1753293707.9167821,50,20,40
1753293729.1217833,50,20,40
1753293751.5176725,50,20,40
1753293772.8645217,50,20,40
1753293794.465694,50,20,40
1753293816.1702642,50,16,32
1753293839.0480645,50,13,26
1753293860.3446264,50,13,26
1753293881.5095074,50,13,26
1753293903.4346936,50,13,26
1753293924.8083725,50,13,26
1753293947.3722692,50,13,26
1753293969.2690017,50,13,26
1753293990.2295816,50,13,26
1753294024.358551,50,13,26
1753294024.3907738,50,13,26
1753294024.4178216,50,13,26
1753294024.481851,50,13,26
1753294024.4895473,50,13,26
1753294024.5156941,50,13,26
1753294024.5515661,50,13,26
1753294110.4441936,50,8,16
1753294123.767267,50,8,16
1753294147.1655746,50,8,16
1753294168.5848012,50,7,14
1753294189.5915787,50,7,14
1753294210.891422,50,7,14
1753294232.1619647,50,7,14
1753294254.4961078,50,7,14
1753294276.3563502,50,7,14
1753294298.8042276,50,7,14
1753294320.5640154,50,7,14
1753294342.9859781,50,7,14
1753294364.7745054,50,7,14
1753294385.333017,50,7,14
1753294406.2527838,50,4,8
1753294428.1623735,50,4,8
1753294449.37779,50,2,4
1753294471.2185073,50,1,2
1753294492.06641,50,1,2
1753294514.6132846,50,1,2
1753294536.415897,50,1,2
1753294557.705412,50,1,2
1753294578.6965952,50,1,2
1753294600.3423672,50,1,2
1753294635.155529,50,1,2
1753294635.1820478,50,1,2
1753294635.2577007,50,1,2
1753294635.427468,50,1,2
1753294635.4694953,50,1,2
1753294635.4839222,50,1,2
1753294635.524986,50,1,2
1753294635.6500554,50,1,2
1753294635.7116306,50,1,2
1753294635.7686453,50,1,2
1753294635.823955,50,1,2
1753294636.0231683,50,1,2
1753294770.6133568,50,0,0
1753294785.2744987,50,0,0
1753294808.5417206,50,0,0
1753294829.864504,50,0,0
1753294850.8704865,50,0,0
1753294873.180558,50,0,0
1753294894.4832935,50,0,0
1753294915.7814085,50,0,0
1753294936.4930608,50,0,0
1753294957.6378458,50,0,0
1753294979.625028,50,0,0
1753295000.8563209,50,0,0
1753295021.5666044,50,0,0
1753295042.227411,50,0,0
1753295063.1466901,50,0,0
1753295084.1619363,50,0,0
1753295105.0336268,50,0,0
1753295125.8544667,50,0,0
1753295146.6970325,50,0,0
1753295167.6693032,50,0,0
1753295188.6878142,50,0,0
1753295209.3180206,50,0,0
1753295243.706071,50,0,0
1753295274.656275,50,0,0
1753295959.390545,50,0,0
1753295988.1362133,50,0,0
1753296017.1895785,50,0,0
1753296045.1150413,50,0,0
1753296073.5645983,50,0,0
1753296102.263583,50,0,0
1753296130.5317557,50,0,0
1753296161.4345942,50,0,0
1753296192.6366224,50,0,0
1753296223.9208312,50,0,0
1753296252.9509504,50,0,0
1753296283.7708523,50,0,0
1753296300.428401,50,0,0
1753296334.469401,50,0,0
1753296363.1934216,50,0,0
1753296396.7888367,50,0,0
1753296425.67543,50,0,0
1753296456.6627264,50,0,0
1753296488.2855892,50,0,0
1753296520.4479654,50,0,0
1753296552.463311,50,0,0
1753296582.4919946,50,0,0
1753296616.0738804,50,0,0
1753296648.332609,50,0,0
1753296682.6596386,50,0,0
1753296716.5515368,50,0,0
1753296886.1219435,50,0,0
1753296916.2239377,50,0,0
1753296945.0308268,50,0,0
1753296973.6629875,50,0,0
1753297003.230053,50,0,0
1753297031.3323238,50,0,0
1753297059.0781524,50,0,0
1753297087.7026434,50,0,0
1753297116.244307,50,0,0
1753297145.0357711,50,0,0
1753297174.6851165,50,0,0
1753297205.211316,50,0,0
1753297234.9644501,50,0,0
1753297264.3082201,50,0,0
1753297293.2573934,50,0,0
1753297322.154067,50,0,0
1753297351.4351213,50,0,0
1753297380.28232,50,0,0
1753297410.4249043,50,0,0
1753297440.3167727,50,0,0
1753297470.5731611,50,0,0
1753297500.041952,50,0,0
1753297529.1768312,50,0,0
1753297558.7596118,50,0,0
1753297629.4494343,50,0,0
1753297659.301387,50,0,0
1753297689.776286,50,0,0
1753297703.0899134,50,0,0
1753297718.9448166,50,0,0
1753297735.6864023,50,0,0
1753297752.9050076,50,0,0
1753297770.1327834,50,0,0
1753297787.2428672,50,0,0
1753297802.6631207,50,0,0
1753297818.3277693,50,0,0
1753297833.6885006,50,0,0
1753297849.3588815,50,0,0
1753297864.9134157,50,0,0
1753297880.5090754,50,0,0
1753297896.1948178,50,0,0
1753297913.717013,50,0,0
1753297930.8079188,50,0,0
1753297948.5436993,50,0,0
1753297965.3743908,50,0,0
1753297982.70024,50,0,0
1753298000.404153,50,0,0
1753298018.108742,50,0,0
1753298034.733061,50,0,0
1753298052.742147,50,0,0
1753298071.0911777,50,0,0
1753298088.5739057,50,0,0
1753298108.166911,50,0,0
1753298125.6144464,50,0,0
1753298143.1898096,50,0,0
1753298160.8018997,50,0,0
1753298178.6988356,50,0,0
1753298196.9687262,50,0,0
1753298215.1725185,50,0,0
1753298232.282277,50,0,0
1753298250.0692337,50,0,0
1753298267.410238,50,0,0
1753298285.7669737,50,0,0
1753298303.4780035,50,0,0
1753298319.6257696,50,0,0
1753298336.3101563,50,0,0
1753298351.1143694,50,0,0
1753298366.226357,50,0,0
1753298383.3524673,50,0,0
1753298398.6402469,50,0,0
1753298414.292398,50,0,0
1753298428.3102334,50,0,0
1753298444.159638,50,0,0
1753298458.9100773,50,0,0
1753298474.0396607,50,0,0
1753298488.9422357,50,0,0
1753298503.9356463,50,0,0
1753298518.507747,50,0,0
1753298534.4075923,50,0,0
1753298534.5369446,50,0,0
1753298534.537629,50,0,0
1753298534.7294676,50,0,0
1753298534.7966251,50,0,0
1753298534.808549,50,0,0
1753298534.8383217,50,0,0
1753298534.853132,50,0,0
1753298534.8536918,50,0,0
1753298534.854803,50,0,0
1753298534.8794684,50,0,0
1753298535.0002258,50,0,0
1753298535.000905,50,0,0
1753298535.0019207,50,0,0
1753298535.0871801,50,0,0
1753298535.1748946,50,0,0
1753298651.3454976,50,1,2
1753298653.504396,50,4,8
1753298653.8239667,50,4,8
1753298654.900921,50,6,12
1753298656.143708,50,12,24
1753298656.7951984,50,14,28
1753298656.820083,50,15,30
1753298656.9799051,50,15,30
1753298657.3279488,50,16,32
1753298657.4706554,50,16,32
1753298657.9301095,50,16,32
1753298657.968901,50,16,32
1753298658.3307316,50,16,32
1753298658.6964629,50,16,32
1753298658.7390718,50,16,32
1753298659.2272048,50,16,32
1753298763.2729023,50,16,32
1753298770.0766041,50,16,32
1753298777.7725153,50,16,32
1753298804.1769004,50,17,34
1753298806.947387,50,17,34
1753298806.9526577,50,17,34
1753298806.9620733,50,17,34
1753298807.8554547,50,17,34
1753298810.171685,50,18,36
1753298812.2306502,50,18,36
1753298817.4727285,50,18,36
1753298818.1006165,50,18,36
1753298818.2123728,50,18,36
1753298818.3417878,50,18,36
1753298818.5791235,50,18,36
1753298834.7693348,50,19,38
1753298916.4430983,50,23,46
1753298916.8744984,50,23,46
1753298917.407999,50,23,46
1753298919.1569498,50,23,46
1753298922.2307642,50,25,50
1753298923.1615639,50,26,52
1753298923.201354,50,26,52
1753298924.2801614,50,26,52
1753298924.6201658,50,26,52
1753298925.995386,50,27,54
1753298926.2386384,50,27,54
1753298927.0017018,50,28,56
1753298927.1304371,50,28,56
1753298929.0360978,50,29,58
1753298929.288373,50,29,58
1753298953.124013,50,29,58
1753299035.9144366,50,30,60
1753299035.9210658,50,30,60
1753299042.4589636,50,33,66
1753299045.4663925,50,34,68
1753299046.1578429,50,34,68
1753299046.8091977,50,34,68
1753299047.7699692,50,34,68
1753299058.1545825,50,34,68
1753299058.297853,50,34,68
1753299060.5558932,50,34,68
1753299060.7063842,50,34,68
1753299061.8662367,50,34,68
1753299062.5547895,50,34,68
1753299062.926925,50,34,68
1753299150.0064263,50,34,68
1753299151.1922395,50,34,68
1753299164.2834182,50,34,68
1753299169.7140772,50,35,70
1753299177.6363716,50,37,74
1753299178.5623207,50,38,76
1753299179.758604,50,39,78
1753299180.260231,50,39,78
1753299180.8987005,50,39,78
1753299181.1952043,50,39,78
1753299181.307374,50,39,78
1753299181.7024124,50,39,78
1753299182.9997053,50,42,84
1753299183.1504397,50,42,84
1753299183.45926,50,42,84
1753299260.5996943,50,43,86
1753299264.0918577,50,44,88
1753299274.9774919,50,45,90
1753299296.67426,50,46,92
1753299301.5680544,50,49,98
1753299302.4682462,50,49,98
1753299302.813186,50,49,98
1753299364.6170442,50,49,98
1753299378.9522665,50,49,98
1753299412.2031357,50,49,98
1753299430.340447,50,49,98
1753299430.491002,50,49,98
1753299430.6969054,50,49,98
1753299452.5061724,50,49,98
1753299473.1576037,50,49,98
1753299493.6630135,50,49,98
1753299515.784284,50,49,98
1753299536.9886894,50,49,98
1753299558.261226,50,49,98
1753299581.2611878,50,49,98
1753299603.4031591,50,49,98
1753299625.1272259,50,49,98
1753299649.0366118,50,49,98
1753299671.0159009,50,49,98
1753299691.8782444,50,49,98
1753299713.4189901,50,49,98
1753299735.0107076,50,49,98
1753299755.8327472,50,49,98
1753299773.9660852,50,49,98
1753299792.2767305,50,49,98
1753299810.5507894,50,49,98
1753299828.275849,50,49,98
1753299848.3888788,50,49,98
1753299867.4909751,50,49,98
1753299886.0975685,50,49,98
1753299905.898203,50,49,98
1753299924.9941666,50,49,98
1753299944.813276,50,49,98
1753299963.5756578,50,49,98
1753299983.0482495,50,49,98
1753300004.1266026,50,49,98
1753300023.0325222,50,49,98
1753300043.256796,50,49,98
1753300064.38989,50,49,98
1753300085.6262631,50,49,98
1753300104.5645435,50,49,98
1753300124.7703798,50,49,98
1753300146.0559804,50,49,98
1753300165.7675169,50,49,98
1753300185.7302122,50,49,98
1753300204.9928007,50,49,98
1753300225.1415093,50,49,98
1753300245.5766284,50,49,98
1753300265.4847445,50,49,98
1753300284.3706305,50,49,98
1753300303.3543825,50,49,98
1753300324.0545945,50,49,98
1753300343.1431527,50,49,98
1753300363.2623107,50,49,98
1753300384.3322985,50,49,98
1753300406.1971526,50,49,98
1753300427.1803007,50,49,98
1753300450.6241047,50,49,98
1753300473.813305,50,49,98
1753300494.5185208,50,49,98
1753300516.8165176,50,49,98
1753300537.8687909,50,49,98
1753300559.1999793,50,49,98
1753300581.18777,50,49,98
1753300602.7581458,50,49,98
1753300624.429832,50,49,98
1753300646.0696857,50,49,98
1753300667.3468993,50,49,98
1753300688.038847,50,49,98
1753300709.5095162,50,49,98
1753300732.2852106,50,49,98
1753300753.3051438,50,49,98
1753300773.2943733,50,49,98
1753300794.8269343,50,49,98
1753300815.9112835,50,49,98
1753300836.9037435,50,49,98
1753300858.29224,50,49,98
1753300879.9668977,50,49,98
1753300902.1797028,50,49,98
1753300923.2936983,50,49,98
1753300945.1549208,50,49,98
1753300967.1225214,50,49,98
1753300989.7654545,50,49,98
1753301011.3130522,50,49,98
1753301032.9579682,50,49,98
1753301055.0452473,50,49,98
1753301076.2931197,50,49,98
1753301098.139546,50,49,98
1753301119.9574456,50,49,98
1753301142.086963,50,49,98
1753301163.4990733,50,49,98
1753301185.337423,50,49,98
1753301206.4414349,50,49,98
1753301227.8700464,50,49,98
1753301249.4174297,50,49,98
1753301270.9120955,50,49,98
1753301292.7007535,50,49,98
1753301313.9820232,50,49,98
1753301335.5619361,50,49,98
1753301357.4443903,50,49,98
1753301379.13859,50,49,98
1753301400.9798524,50,49,98
1753301423.064335,50,49,98
1753301444.3704512,50,49,98
1753301465.953659,50,49,98
1753301488.4849308,50,49,98
1753301509.7630405,50,49,98
1753301530.823105,50,49,98
1753301553.6544023,50,49,98
1753301575.1369905,50,49,98
1753301597.331713,50,49,98
1753301618.9955595,50,49,98
1753301640.1695135,50,49,98
1753301662.7319992,50,49,98
1753301684.1791873,50,49,98
1753301704.7545192,50,49,98
1753301726.5503232,50,49,98
1753301748.2616973,50,49,98
1753301771.6743762,50,49,98
1753301793.5414116,50,49,98
1753301814.844772,50,49,98
1753301837.1216147,50,49,98
1753301858.590955,50,49,98
1753301880.7351635,50,49,98
1753301900.1075864,50,49,98
1753301920.5984285,50,49,98
1753301939.4451623,50,48,96
1753301953.487359,50,48,96
1753301986.072363,50,48,96
1753302001.8607235,50,48,96
1753302022.8548112,50,48,96
1753302043.642634,50,48,96
1753302065.5456133,50,48,96
1753302087.3464851,50,48,96
1753302109.2535815,50,48,96
1753302131.881703,50,48,96
1753302151.1694026,50,47,94
1753302164.3924885,50,47,94
1753302196.9320219,50,47,94
1753302211.993181,50,47,94
1753302231.7832608,50,47,94
1753302253.5846949,50,47,94
1753302275.4974768,50,47,94
1753302297.4488153,50,47,94
1753302319.9683435,50,38,76
1753302341.0272956,50,32,64
1753302362.0279045,50,32,64
1753302382.9543967,50,31,62
1753302396.1584346,50,31,62
1753302427.6726863,50,30,60
1753302442.0697825,50,30,60
1753302462.93597,50,28,56
1753302484.16228,50,28,56
1753302505.471451,50,28,56
1753302527.8021102,50,28,56
1753302548.973141,50,20,40
1753302571.5764194,50,20,40
1753302592.8642557,50,20,40
1753302615.033294,50,20,40
1753302637.0004444,50,18,36
1753302656.9728224,50,18,36
1753302677.2533596,50,18,36
1753302698.3974152,50,17,34
1753302719.1224456,50,14,28
1753302738.796963,50,14,28
1753302758.108394,50,14,28
1753302778.3131852,50,14,28
1753302798.871877,50,14,28
1753302819.162257,50,7,14
1753302839.5344563,50,7,14
1753302860.486625,50,7,14
1753302895.1574562,50,5,10
1753302895.2441378,50,5,10
1753302895.4884577,50,5,10
1753302895.723886,50,5,10
1753302895.7958841,50,5,10
1753302895.9102972,50,5,10
1753302896.0033333,50,5,10
1753302896.027026,50,5,10
1753302896.0881486,50,5,10
1753302896.166203,50,5,10
1753302896.2781572,50,5,10
1753302896.3347735,50,5,10
1753302896.3514307,50,5,10
1753302896.4120796,50,5,10
1753302896.4130936,50,5,10
1753302896.4453726,50,5,10
1753302896.5065365,50,5,10
1753302896.5165024,50,5,10
1753302896.625741,50,5,10
1753302896.6405194,50,5,10
1753302896.8730333,50,5,10
1753302896.9653928,50,5,10
1753302897.0642385,50,5,10
1753302897.12037,50,5,10
1753302897.2873313,50,5,10
1753302897.4302194,50,5,10
1753302897.6223612,50,5,10
1753302897.989532,50,5,10
1753302898.0309372,50,5,10
1753302898.185343,50,5,10
1753302898.2828507,50,5,10
1753302898.5394533,50,5,10
1753303149.757917,50,0,0
1753303150.1464562,50,0,0
1753303150.6685348,50,0,0
1753303150.6800768,50,0,0
1753303150.7206185,50,0,0
1753303150.824925,50,0,0
1753303150.8503327,50,0,0
1753303174.6046207,50,0,0
1753303176.419175,50,0,0
1753303257.4854226,50,0,0
1753303271.6522636,50,0,0
1753303291.8427992,50,0,0
1753303312.494511,50,0,0
1753303331.6394243,50,0,0
1753303351.9456968,50,0,0
1753303372.6630685,50,0,0
1753303392.1554754,50,0,0
1753303412.4406714,50,0,0
1753303433.5327246,50,0,0
1753303453.2893846,50,0,0
1753303472.205715,50,0,0
1753303505.798721,50,0,0
1753303505.8488226,50,0,0
1753303505.8726585,50,0,0
1753303505.8915548,50,0,0
1753303505.924418,50,0,0
1753303562.0329325,50,0,0
1753304075.8631122,50,0,0
1753304101.6804094,50,0,0
1753304127.4468827,50,0,0
1753304158.4892411,50,0,0
1753304174.8284779,50,0,0
1753304210.0546944,50,0,0
1753304227.0192523,50,0,0
1753304261.5897553,50,0,0
1753304297.3281066,50,0,0
1753304314.515454,50,0,0
1753304330.3199449,50,0,0
1753304368.3916867,50,0,0
1753304403.8900142,50,0,0
1753304438.676609,50,0,0
1753304455.8415108,50,0,0
1753304490.624722,50,0,0
1753304525.6581376,50,0,0
1753304561.7091398,50,0,0
1753304597.8612502,50,0,0
1753304634.342848,50,0,0
1753304671.8901002,50,0,0
1753304707.686877,50,0,0
1753304742.5693734,50,0,0
1753304776.454004,50,0,0
1753304793.4976869,50,0,0
1753304827.9867482,50,0,0
1753304862.63894,50,0,0
1753304897.9334767,50,0,0
1753304932.7893424,50,0,0
1753304970.857906,50,0,0
1753305008.3457203,50,0,0
1753305025.7947412,50,0,0
1753305061.0055773,50,0,0
1753305099.68392,50,0,0
1753305134.5792484,50,0,0
1753305171.9807417,50,0,0
1753305208.3843303,50,0,0
1753305244.9171774,50,0,0
1753305280.0488734,50,0,0
1753305316.7803736,50,0,0
1753305352.3571033,50,0,0
1753305388.913998,50,0,0
1753305427.0129583,50,0,0
1753305465.1209204,50,0,0
1753305502.2339356,50,0,0
1753305540.4414873,50,0,0
1753305577.5022802,50,0,0
1753305615.377691,50,0,0
1753305653.949008,50,0,0
1753305691.4574146,50,0,0
1753305835.4700854,50,0,0
1753305873.9967854,50,0,0
1753305911.9558249,50,0,0
1753305951.1717157,50,0,0
1753305968.9544601,50,0,0
1753306008.2349107,50,0,0
1753306026.368521,50,0,0
1753306093.3608057,50,0,0
1753306131.8206928,50,0,0
1753306168.8626704,50,0,0
1753306187.1335437,50,0,0
1753306208.3239813,50,0,0
1753306229.549695,50,0,0
1753306250.7159922,50,0,0
1753306271.2777627,50,0,0
1753306291.4190485,50,0,0
1753306312.2970228,50,0,0
1753306333.2782037,50,0,0
1753306354.951413,50,0,0
1753306375.8508375,50,0,0
1753306396.5244837,50,0,0
1753306416.783396,50,0,0
1753306438.0223954,50,0,0
1753306459.0048735,50,0,0
1753306506.5212193,50,0,0
1753306527.582588,50,0,0
1753306547.7841942,50,0,0
1753306568.854394,50,0,0
1753306589.7574885,50,0,0
1753306610.8469903,50,0,0
1753306631.4930527,50,0,0
1753306652.6411617,50,0,0
1753306672.6902096,50,0,0
1753306693.444893,50,0,0
1753306714.4017184,50,0,0
1753306735.111042,50,0,0
1753306755.6242878,50,0,0
1753306777.5816061,50,0,0
1753306799.6070447,50,0,0
1753306820.119401,50,0,0
1753306841.5623655,50,0,0
1753306861.5874455,50,0,0
1753306881.5679793,50,0,0
1753306902.267736,50,0,0
1753306922.1337953,50,0,0
1753306943.3722095,50,0,0
1753306964.4694195,50,0,0
1753306985.2410448,50,0,0
1753307005.4217823,50,0,0
1753307028.7292564,50,0,0
1753307049.6453812,50,0,0
1753307070.3952103,50,0,0
1753307093.7660112,50,0,0
1753307114.653374,50,0,0
1753307135.6121566,50,0,0
1753307158.8101656,50,0,0
1753307179.6238263,50,0,0
1753307200.3439221,50,0,0
1753307221.575488,50,0,0
1753307242.2616673,50,0,0
1753307264.5621989,50,0,0
1753307264.6867692,50,0,0
1753307264.920469,50,0,0
1753307264.9609294,50,0,0
1753307265.008024,50,0,0
1753307265.0340075,50,0,0
1753307265.0349889,50,0,0
1753307265.073296,50,0,0
1753307265.103202,50,0,0
1753307265.1924787,50,0,0
1753307265.1989522,50,0,0
1753307265.2636034,50,0,0
1753307265.275862,50,0,0
1753307265.2765386,50,0,0
1753307265.6151373,50,0,0
1753307265.6524036,50,0,0
1753307434.5889592,50,3,6
1753307435.645388,50,8,16
1753307436.2977211,50,10,20
1753307436.8969128,50,12,24
1753307437.1443465,50,12,24
1753307437.5180395,50,13,26
1753307437.6145542,50,13,26
1753307438.066823,50,13,26
1753307438.1926486,50,13,26
1753307438.4163287,50,14,28
1753307438.9392068,50,14,28
1753307439.0477698,50,15,30
1753307439.6528757,50,15,30
1753307440.51926,50,16,32
1753307441.1737661,50,16,32
1753307441.578992,50,16,32
1753307653.836434,50,16,32
1753307653.8491123,50,16,32
1753307653.9409072,50,16,32
1753307654.2027104,50,16,32
1753307654.3033662,50,16,32
1753307654.3646803,50,16,32
1753307654.5763175,50,16,32
1753307654.6294904,50,16,32
1753307654.6707623,50,16,32
1753307657.3694003,50,16,32
1753307657.377581,50,16,32
1753307657.786234,50,16,32
1753307658.6041539,50,16,32
1753307665.4779096,50,16,32
1753307665.8808756,50,16,32
1753307666.9211562,50,16,32
1753307838.7556307,50,17,34
1753307840.4422193,50,18,36
1753307849.5541854,50,21,42
1753307850.168637,50,22,44
1753307851.0713353,50,25,50
1753307851.843264,50,29,58
1753307852.985237,50,32,64
1753307853.3711386,50,32,64
1753307853.448052,50,32,64
1753307853.6296766,50,32,64
1753307854.1770027,50,32,64
1753307854.5415552,50,32,64
1753307854.71728,50,32,64
1753307854.8852215,50,32,64
1753307855.0727143,50,32,64
1753307855.3371131,50,32,64
1753308006.1996677,50,31,62
1753308052.0398846,50,31,62
1753308098.146548,50,32,64
1753308100.0493646,50,32,64
1753308102.0900426,50,32,64
1753308105.5442655,50,32,64
1753308105.9319067,50,32,64
1753308115.4181128,50,32,64
1753308119.413103,50,32,64
1753308120.1808693,50,32,64
1753308122.3917418,50,32,64
1753308123.1645706,50,32,64
1753308123.661462,50,32,64
1753308124.4722056,50,32,64
1753308124.9108915,50,32,64
1753308124.9497375,50,32,64
1753308125.6178536,50,32,64
1753308129.2859185,50,33,66
1753308296.73478,50,35,70
1753308297.1432464,50,35,70
1753308302.5615878,50,37,74
1753308303.8239825,50,37,74
1753308314.5679688,50,39,78
1753308314.6220388,50,39,78
1753308317.1709547,50,42,84
1753308319.406397,50,45,90
1753308320.1729035,50,46,92
1753308320.202897,50,46,92
1753308321.2184844,50,46,92
1753308321.2417438,50,46,92
1753308321.2775924,50,46,92
1753308322.3345094,50,46,92
1753308322.4192588,50,46,92
1753308339.862586,50,46,92
1753308537.543635,50,48,96
1753308537.9684944,50,48,96
1753308635.9050465,50,47,94
1753308636.0166004,50,47,94
1753308636.0425315,50,47,94
1753308673.4674885,50,46,92
1753308677.235827,50,46,92
1753308741.5570369,50,46,92
1753308764.7129233,50,46,92
1753308787.9886084,50,46,92
1753308837.061508,50,46,92
1753308858.9323115,50,46,92
1753308859.0982347,50,46,92
1753308859.37285,50,46,92
1753308889.6079612,50,46,92
1753308918.8762808,50,46,92
1753308948.7426271,50,46,92
1753308978.9720533,50,46,92
1753309008.5151956,50,46,92
1753309037.8219953,50,46,92
1753309067.5417793,50,46,92
1753309097.706001,50,46,92
1753309127.2071111,50,46,92
1753309158.8182502,50,46,92
1753309188.8660424,50,46,92
1753309218.591453,50,46,92
1753309247.3408587,50,46,92
1753309276.4578676,50,46,92
1753309307.4548116,50,46,92
1753309337.323259,50,46,92
1753309367.0871506,50,46,92
1753309396.7179482,50,46,92
1753309426.027345,50,46,92
1753309456.5322268,50,46,92
1753309487.054871,50,46,92
1753309517.780019,50,46,92
1753309549.9953053,50,46,92
1753309579.9352329,50,46,92
1753309609.6115644,50,46,92
1753309639.87859,50,46,92
1753309668.6728587,50,46,92
1753309698.8198364,50,46,92
1753309728.230826,50,46,92
1753309758.7884738,50,46,92
1753309789.360669,50,46,92
1753309819.265359,50,46,92
1753309848.4968617,50,46,92
1753309876.425576,50,46,92
1753309903.6503413,50,46,92
1753309930.5499556,50,46,92
1753309955.2668865,50,45,90
1753309974.8894498,50,45,90
1753310018.7126615,50,45,90
1753310038.574649,50,45,90
1753310065.8500228,50,45,90
1753310092.2929604,50,45,90
1753310119.925904,50,45,90
1753310146.9036486,50,45,90
1753310173.716429,50,45,90
1753310201.5950413,50,45,90
1753310228.388695,50,45,90
1753310255.4888155,50,45,90
1753310282.9177177,50,45,90
1753310309.8822348,50,45,90
1753310337.290227,50,45,90
1753310365.581807,50,45,90
1753310392.9880128,50,45,90
1753310419.8062084,50,45,90
1753310447.4761484,50,45,90
1753310474.309155,50,45,90
1753310501.6751943,50,45,90
1753310529.3639622,50,45,90
1753310556.5761926,50,45,90
1753310582.922929,50,45,90
1753310611.2702937,50,45,90
1753310637.8213558,50,45,90
1753310666.659116,50,45,90
1753310692.1310892,50,44,88
1753310713.862943,50,44,88
1753310761.4877412,50,44,88
1753310781.1225786,50,44,88
1753310809.0791478,50,44,88
1753310838.7995453,50,44,88
1753310866.5816784,50,44,88
1753310891.5463593,50,43,86
1753310912.1700718,50,43,86
1753310956.8832948,50,43,86
1753310983.2468522,50,43,86
1753311011.2773116,50,43,86
1753311039.6302202,50,43,86
1753311068.543995,50,43,86
1753311096.9060633,50,33,66
1753311131.068727,50,29,58
1753311159.9356642,50,29,58
1753311187.6336977,50,29,58
1753311228.5784822,50,29,58
1753311255.9179575,50,29,58
1753311297.456491,50,29,58
1753311297.553387,50,29,58
1753311297.6275675,50,29,58
1753311297.6566098,50,29,58
1753311297.9362977,50,29,58
1753311298.000941,50,29,58
1753311298.0185757,50,29,58
1753311298.1365347,50,29,58
1753311298.165926,50,29,58
1753311298.2318156,50,29,58
1753311298.2554877,50,29,58
1753311298.2816432,50,29,58
1753311298.4005275,50,29,58
1753311298.695676,50,29,58
1753311568.4269755,50,16,32
1753311588.9982822,50,16,32
1753311617.0754635,50,16,32
1753311644.648983,50,16,32
1753311671.5705194,50,16,32
1753311699.4826322,50,16,32
1753311728.496239,50,16,32
1753311755.3957624,50,15,30
1753311783.495323,50,14,28
1753311819.1248565,50,14,28
1753311847.6862955,50,14,28
1753311885.2313654,50,14,28
1753311912.116659,50,14,28
1753311942.8243425,50,11,22
1753311971.6736472,50,2,4
1753311999.713235,50,2,4
1753312026.522515,50,2,4
1753312053.8467827,50,2,4
1753312082.3404872,50,2,4
1753312109.468229,50,2,4
1753312136.9345572,50,2,4
1753312164.0028043,50,0,0
1753312191.586051,50,0,0
1753312220.8487957,50,0,0
1753312249.3968282,50,0,0
1753312278.278375,50,0,0
1753312305.2075334,50,0,0
1753312332.2155375,50,0,0
1753312361.0747097,50,0,0
1753312387.9479825,50,0,0
1753312415.8343563,50,0,0
1753312442.8898022,50,0,0
1753312469.0369132,50,0,0
1753312510.3518312,50,0,0
1753312510.3975303,50,0,0
1753312510.5154629,50,0,0
1753312510.5645447,50,0,0
1753312510.620324,50,0,0
1753312510.631617,50,0,0
1753312510.7589962,50,0,0
1753312510.862626,50,0,0
1753312510.8676593,50,0,0
1753312510.8891563,50,0,0
1753312510.9464223,50,0,0
1753312511.2050433,50,0,0
1753312511.240777,50,0,0
1753312511.494621,50,0,0
1753312694.901881,50,0,0
1753313486.5232441,50,0,0
1753313526.656245,50,0,0
1753313565.8316333,50,0,0
1753313604.857461,50,0,0
1753313644.5585604,50,0,0
1753313684.7239823,50,0,0
1753313704.2253833,50,0,0
1753313743.0783129,50,0,0
1753313783.2869468,50,0,0
1753313822.4636946,50,0,0
1753313862.1430845,50,0,0
1753313901.5523741,50,0,0
1753313941.2350411,50,0,0
1753313960.1987774,50,0,0
1753313998.885028,50,0,0
1753314017.9519904,50,0,0
1753314056.8230848,50,0,0
1753314097.684071,50,0,0
1753314134.4332278,50,0,0
1753314172.5999029,50,0,0
1753314210.503103,50,0,0
1753314246.6585488,50,0,0
1753314284.2893875,50,0,0
1753314322.925287,50,0,0
1753314359.4980457,50,0,0
1753314397.5276403,50,0,0
1753314434.9318922,50,0,0
1753314473.8118052,50,0,0
1753314511.1158497,50,0,0
1753314550.1739466,50,0,0
1753314590.0351322,50,0,0
1753314627.4314573,50,0,0
1753314666.113192,50,0,0
1753314703.4534593,50,0,0
1753314741.2791078,50,0,0
1753314777.7842283,50,0,0
1753314814.9284344,50,0,0
1753314852.931189,50,0,0
1753314890.4762614,50,0,0
1753314929.2429178,50,0,0
1753314967.8521383,50,0,0
1753315007.1382692,50,0,0
1753315045.0806892,50,0,0
1753315082.206501,50,0,0
1753315115.9364088,50,0,0
1753315149.1599522,50,0,0
1753315181.8394082,50,0,0
1753315217.1457632,50,0,0
1753315233.7913115,50,0,0
1753315266.3508823,50,0,0
1753315349.357067,50,0,0
1753315384.469318,50,0,0
1753315417.6836734,50,0,0
1753315450.431272,50,0,0
1753315504.9446747,50,0,0
1753315537.9861162,50,0,0
1753315552.9244044,50,0,0
1753315570.1809096,50,0,0
1753315587.9738624,50,0,0
1753315606.1898248,50,0,0
1753315623.2829902,50,0,0
1753315639.7717402,50,0,0
1753315657.3372953,50,0,0
1753315676.3569534,50,0,0
1753315695.7448962,50,0,0
1753315712.643724,50,0,0
1753315729.7343497,50,0,0
1753315746.2436917,50,0,0
1753315761.4003315,50,0,0
1753315777.9125242,50,0,0
1753315794.2710855,50,0,0
1753315809.8385015,50,0,0
1753315825.3429813,50,0,0
1753315843.502923,50,0,0
1753315860.571621,50,0,0
1753315876.994985,50,0,0
1753315893.2728512,50,0,0
1753315910.9794958,50,0,0
1753315927.8889713,50,0,0
1753315943.1573818,50,0,0
1753315959.1745512,50,0,0
1753315977.0652518,50,0,0
1753315995.0767703,50,0,0
1753316010.765186,50,0,0
1753316026.8739932,50,0,0
1753316042.5675588,50,0,0
1753316061.0179172,50,0,0
1753316077.4161928,50,0,0
1753316094.421585,50,0,0
1753316111.416943,50,0,0
1753316128.7622628,50,0,0
1753316145.4357452,50,0,0
1753316161.2753422,50,0,0
1753316177.9824936,50,0,0
1753316193.8684602,50,0,0
1753316209.2051623,50,0,0
1753316224.7825193,50,0,0
1753316241.1984816,50,0,0
1753316257.786707,50,0,0
1753316272.0991817,50,0,0
1753316288.3227568,50,0,0
1753316305.0836787,50,0,0
1753316320.52088,50,0,0
1753316337.188392,50,0,0
1753316352.472804,50,0,0
1753316368.8942013,50,0,0
1753316385.5571363,50,0,0
1753316385.650187,50,0,0
1753316385.76083,50,0,0
1753316385.8014083,50,0,0
1753316385.820791,50,0,0
1753316385.860495,50,0,0
1753316385.8771398,50,0,0
1753316385.8775792,50,0,0
1753316385.8780925,50,0,0
1753316385.8818665,50,0,0
1753316385.8958747,50,0,0
1753316385.9064496,50,0,0
1753316385.925073,50,0,0
1753316385.9636557,50,0,0
1753316385.969392,50,0,0
1753316386.1029732,50,0,0
1753316515.963744,50,4,8
1753316516.307339,50,6,12
1753316516.845626,50,6,12
1753316517.0598123,50,6,12
1753316517.3384647,50,6,12
1753316517.3398495,50,6,12
1753316521.5283778,50,8,16
1753316522.4399521,50,10,20
1753316523.9711988,50,13,26
1753316524.1067681,50,13,26
1753316524.940892,50,16,32
1753316525.6830976,50,16,32
1753316525.858949,50,16,32
1753316525.8853724,50,16,32
1753316526.2701852,50,16,32
1753316526.416919,50,16,32
1753316675.459493,50,16,32
1753316675.7581215,50,16,32
1753316675.8975844,50,16,32
1753316675.9238975,50,16,32
1753316676.1327734,50,16,32
1753316677.8811522,50,16,32
1753316678.0752583,50,16,32
1753316678.3484216,50,16,32
1753316678.3495724,50,16,32
1753316678.6277792,50,16,32
1753316678.7773356,50,16,32
1753316678.9836314,50,16,32
1753316679.4258702,50,16,32
1753316680.6516085,50,16,32
1753316682.0337734,50,16,32
1753316682.2600057,50,16,32
1753316817.2413168,50,19,38
1753316817.5380802,50,19,38
1753316817.8842647,50,19,38
1753316824.372537,50,20,40
1753316826.430694,50,24,48
1753316826.6555626,50,24,48
1753316826.889867,50,24,48
1753316827.3262405,50,26,52
1753316828.8213623,50,32,64
1753316828.8516316,50,32,64
1753316829.3035703,50,32,64
1753316829.3620312,50,32,64
1753316829.5058186,50,32,64
1753316830.0505607,50,32,64
1753316830.4758317,50,32,64
1753316830.7838888,50,32,64
1753316944.0011687,50,32,64
1753316944.127449,50,32,64
1753316946.1283789,50,32,64
1753317004.8181422,50,32,64
1753317008.0211647,50,32,64
1753317008.064211,50,32,64
1753317008.2881253,50,32,64
1753317008.351343,50,32,64
1753317011.4683115,50,32,64
1753317012.6204026,50,32,64
1753317013.7707617,50,32,64
1753317017.5014887,50,32,64
1753317018.0484035,50,32,64
1753317018.1764693,50,32,64
1753317018.5282803,50,32,64
1753317018.5611537,50,32,64
1753317023.117232,50,33,66
1753317025.9721882,50,34,68
1753317028.8132246,50,35,70
1753317147.5264125,50,37,74
1753317148.3917816,50,37,74
1753317151.8831873,50,39,78
1753317153.3973708,50,39,78
1753317155.0744169,50,41,82
1753317156.763575,50,42,84
1753317157.5455213,50,43,86
1753317158.7575448,50,43,86
1753317163.0097687,50,44,88
1753317165.078822,50,48,96
1753317165.764368,50,48,96
1753317166.7725656,50,48,96
1753317168.4007778,50,48,96
1753317185.2932832,50,48,96
1753317185.7515655,50,48,96
1753317320.9200115,50,50,100
1753317322.62923,50,50,100
1753317359.5192955,50,50,100
1753317359.6679013,50,50,100
1753317359.8745668,50,50,100
1753317386.4142945,50,50,100
1753317409.8291106,50,50,100
1753317433.9511702,50,50,100
1753317457.6806967,50,50,100
1753317478.2236528,50,49,98
1753317495.910704,50,49,98
1753317533.9985592,50,49,98
1753317550.5201728,50,49,98
1753317574.5097961,50,49,98
1753317598.9128878,50,49,98
1753317621.7168055,50,49,98
1753317646.3344586,50,49,98
1753317670.930965,50,49,98
1753317694.4664383,50,49,98
1753317717.7624636,50,49,98
1753317743.859913,50,49,98
1753317769.1349642,50,49,98
1753317791.2857857,50,49,98
1753317816.362467,50,49,98
1753317839.3751748,50,49,98
1753317861.588889,50,49,98
1753317885.4433692,50,49,98
1753317908.7865996,50,49,98
1753317933.4182348,50,49,98
1753317956.9669664,50,49,98
1753317979.620208,50,49,98
1753318004.6593828,50,49,98
1753318027.415105,50,49,98
1753318051.469894,50,49,98
1753318079.8889606,50,49,98
1753318103.088194,50,49,98
1753318127.0915759,50,49,98
1753318153.1633577,50,49,98
1753318176.1243548,50,49,98
1753318199.5436835,50,49,98
1753318225.3034246,50,49,98
1753318248.7896304,50,49,98
1753318274.1796505,50,49,98
1753318297.949654,50,49,98
1753318321.5911849,50,49,98
1753318346.522096,50,49,98
1753318369.0265603,50,49,98
1753318392.4605381,50,49,98
1753318416.0348992,50,49,98
1753318440.5370014,50,49,98
1753318463.991717,50,49,98
1753318489.7061653,50,49,98
1753318513.9483705,50,49,98
1753318536.572636,50,49,98
1753318561.1758804,50,49,98
1753318585.381112,50,49,98
1753318609.4945638,50,49,98
1753318633.2535493,50,49,98
1753318656.076538,50,49,98
1753318681.084694,50,49,98
1753318705.252726,50,49,98
1753318727.7258024,50,49,98
1753318751.882425,50,49,98
1753318776.3514123,50,49,98
1753318799.408843,50,49,98
1753318824.3458652,50,49,98
1753318848.2023993,50,49,98
1753318872.6770105,50,49,98
1753318897.3472593,50,49,98
1753318921.1891823,50,49,98
1753318945.5847437,50,49,98
1753318967.7710085,50,49,98
1753318992.1571987,50,49,98
1753319015.1000397,50,49,98
1753319037.8123925,50,49,98
1753319066.2648208,50,49,98
1753319089.1657841,50,49,98
1753319112.8014333,50,49,98
1753319137.316365,50,49,98
1753319161.638937,50,49,98
1753319184.9026392,50,49,98
1753319210.2092557,50,49,98
1753319232.878677,50,49,98
1753319256.125037,50,49,98
1753319283.0438411,50,49,98
1753319306.3117719,50,49,98
1753319329.0575867,50,49,98
1753319352.9145036,50,49,98
1753319374.2313242,50,49,98
1753319396.6134956,50,49,98
1753319419.4942589,50,49,98
1753319445.0241573,50,49,98
1753319467.0108814,50,49,98
1753319490.9848132,50,49,98
1753319510.7978516,50,48,96
1753319526.0750058,50,48,96
1753319563.3002193,50,48,96
1753319582.003988,50,48,96
1753319605.252196,50,48,96
1753319629.19019,50,48,96
1753319652.802842,50,48,96
1753319677.4016144,50,48,96
1753319702.4471688,50,48,96
1753319725.4891567,50,48,96
1753319751.6475673,50,48,96
1753319777.2542045,50,48,96
1753319802.4518802,50,48,96
1753319825.0556908,50,47,94
1753319840.537484,50,47,94
1753319877.9941525,50,47,94
1753319893.6172147,50,47,94
1753319919.3426194,50,47,94
1753319942.8395567,50,47,94
1753319966.9362469,50,47,94
1753319990.4642797,50,47,94
1753320012.932737,50,47,94
1753320037.0634553,50,47,94
1753320062.3419116,50,47,94
1753320085.7793145,50,47,94
1753320111.6711748,50,47,94
1753320135.9612765,50,47,94
1753320160.8592389,50,37,74
1753320186.3981917,50,31,62
1753320209.9866946,50,31,62
1753320233.0679028,50,31,62
1753320258.4090295,50,31,62
1753320283.7198107,50,31,62
1753320308.1734464,50,31,62
1753320332.95144,50,31,62
1753320358.0104585,50,31,62
1753320381.9866166,50,31,62
1753320405.4687562,50,31,62
1753320427.7682762,50,31,62
1753320452.2830567,50,21,42
1753320476.9578044,50,17,34
1753320501.6975992,50,15,30
1753320525.7402017,50,15,30
1753320548.9247167,50,15,30
1753320575.3375432,50,15,30
1753320598.037301,50,15,30
1753320621.667272,50,15,30
1753320647.2096303,50,15,30
1753320687.7502706,50,12,24
1753320687.7712674,50,12,24
1753320687.8683746,50,12,24
1753320687.9216337,50,12,24
1753320688.1338937,50,12,24
1753320688.2313724,50,12,24
1753320688.4387567,50,12,24
1753320688.4581754,50,12,24
1753320688.5158696,50,12,24
1753320688.5411098,50,12,24
1753320688.648157,50,12,24
1753320688.762287,50,12,24
1753320688.879348,50,12,24
1753320688.912262,50,12,24
1753320688.920741,50,12,24
1753320689.0829437,50,12,24
1753320689.156863,50,12,24
1753320689.1923206,50,12,24
1753320689.2296069,50,12,24
1753320689.5727952,50,12,24
1753320689.6049948,50,12,24
1753320689.635141,50,12,24
1753320689.6414561,50,12,24
1753320689.9370918,50,12,24
1753320689.9750016,50,12,24
1753320689.987842,50,12,24
1753320690.1234324,50,12,24
1753320690.2712033,50,12,24
1753320690.575869,50,12,24
1753320690.963581,50,12,24
1753320691.0064018,50,12,24
1753320691.6704817,50,12,24
1753321013.8480926,50,0,0
1753321013.8485866,50,0,0
1753321013.8495433,50,0,0
1753321120.3906548,50,0,0
1753321140.9113789,50,0,0
1753321169.9250245,50,0,0
1753321200.5952194,50,0,0
1753321228.8592968,50,0,0
1753321258.1395876,50,0,0
1753321299.518931,50,0,0
1753321299.6074693,50,0,0
1753321299.7289872,50,0,0
1753321299.7888472,50,0,0
1753321299.8314652,50,0,0
1753321299.9764986,50,0,0
1753321299.9821186,50,0,0
1753321299.9991717,50,0,0
1753321300.0523512,50,0,0
1753321300.118348,50,0,0
1753321300.2246,50,0,0
1753321300.3229876,50,0,0
1753321482.1396282,50,0,0
1753322326.4887896,50,0,0
1753322369.6473022,50,0,0
1753322413.6091936,50,0,0
1753322458.338724,50,0,0
1753322479.6239786,50,0,0
1753322522.948264,50,0,0
1753322566.181903,50,0,0
1753322608.444996,50,0,0
1753322656.0981066,50,0,0
1753322701.7940042,50,0,0
1753322746.0958936,50,0,0
1753322765.6942554,50,0,0
1753322786.534911,50,0,0
1753322807.272011,50,0,0
1753322851.1848133,50,0,0
1753322895.2083685,50,0,0
1753322937.8134053,50,0,0
1753322981.3185308,50,0,0
1753323024.0709636,50,0,0
1753323068.6447227,50,0,0
1753323110.8710113,50,0,0
1753323132.34562,50,0,0
1753323151.628713,50,0,0
1753323172.2335422,50,0,0
1753323215.386795,50,0,0
1753323258.389066,50,0,0
1753323279.0649116,50,0,0
1753323321.361419,50,0,0
1753323363.4438334,50,0,0
1753323406.9419644,50,0,0
1753323448.7393816,50,0,0
1753323491.4338868,50,0,0
1753323535.203964,50,0,0
1753323578.0331607,50,0,0
1753323620.9948342,50,0,0
1753323663.9522455,50,0,0
1753323707.3802373,50,0,0
1753323750.7743673,50,0,0
1753323794.9894261,50,0,0
1753323840.149296,50,0,0
1753323883.4858613,50,0,0
1753323927.0721316,50,0,0
1753323971.3831384,50,0,0
1753324015.2582028,50,0,0
1753324035.148465,50,0,0
1753324079.128465,50,0,0
1753324121.4099462,50,0,0
1753324162.8736482,50,0,0
1753324206.747974,50,0,0
1753324250.3104308,50,0,0
1753324412.7533715,50,0,0
1753324456.743652,50,0,0
1753324500.3959794,50,0,0
1753324543.7133365,50,0,0
1753324588.845199,50,0,0
1753324608.929432,50,0,0
1753324653.4509892,50,0,0
1753324697.9147081,50,0,0
1753324741.9724693,50,0,0
1753324819.4434135,50,0,0
1753324864.225955,50,0,0
1753324883.8687236,50,0,0
1753324907.1652467,50,0,0
1753324930.264862,50,0,0
1753324953.2766848,50,0,0
1753324977.4673772,50,0,0
1753324999.6879864,50,0,0
1753325023.230786,50,0,0
1753325047.0976057,50,0,0
1753325070.5707753,50,0,0
1753325094.1775515,50,0,0
1753325116.7091954,50,0,0
1753325140.1474383,50,0,0
1753325163.840252,50,0,0
1753325187.5193913,50,0,0
1753325211.4621658,50,0,0
1753325235.2625797,50,0,0
1753325258.238042,50,0,0
1753325281.7306387,50,0,0
1753325306.4617085,50,0,0
1753325329.7142863,50,0,0
1753325352.562525,50,0,0
1753325375.3221557,50,0,0
1753325398.8990278,50,0,0
1753325421.9185963,50,0,0
1753325444.2834878,50,0,0
1753325469.5609994,50,0,0
1753325492.2881145,50,0,0
1753325515.8965209,50,0,0
1753325540.5944211,50,0,0
1753325563.9910128,50,0,0
1753325587.6344643,50,0,0
1753325610.4689913,50,0,0
1753325633.703274,50,0,0
1753325658.3976865,50,0,0
1753325682.6704292,50,0,0
1753325710.5471766,50,0,0
1753325734.3199942,50,0,0
1753325761.8936899,50,0,0
1753325785.1926317,50,0,0
1753325807.7629826,50,0,0
1753325831.4479692,50,0,0
1753325854.906358,50,0,0
1753325877.720594,50,0,0
1753325900.996529,50,0,0
1753325925.1137311,50,0,0
1753325948.9256926,50,0,0
1753325973.026834,50,0,0
1753325995.7322848,50,0,0
1753326019.4485724,50,0,0
1753326041.5999508,50,0,0
1753326066.0697453,50,0,0
1753326066.1053627,50,0,0
1753326066.2307794,50,0,0
1753326066.2875397,50,0,0
1753326066.3400776,50,0,0
1753326066.3497262,50,0,0
1753326066.3784175,50,0,0
1753326066.3917842,50,0,0
1753326066.416276,50,0,0
1753326066.463233,50,0,0
1753326066.522053,50,0,0
1753326066.5718024,50,0,0
1753326066.6973717,50,0,0
1753326066.7440338,50,0,0
1753326066.8581893,50,0,0
1753326067.5728884,50,0,0
1753326264.8423471,50,6,12
1753326264.9764128,50,7,14
1753326265.3958972,50,7,14
1753326265.5871806,50,7,14
1753326265.8745859,50,8,16
1753326266.4714806,50,9,18
1753326266.724365,50,9,18
1753326267.171895,50,9,18
1753326267.327321,50,9,18
1753326271.9312315,50,13,26
1753326272.9525378,50,16,32
1753326274.305453,50,16,32
1753326274.988832,50,16,32
1753326275.174532,50,16,32
1753326275.3301127,50,16,32
1753326275.8748422,50,16,32
1753326518.8703828,50,15,30
1753326519.0547159,50,15,30
1753326519.1095426,50,15,30
1753326519.3830903,50,15,30
1753326519.8302894,50,15,30
1753326520.7052448,50,15,30
1753326520.7530794,50,15,30
1753326521.323571,50,15,30
1753326523.314723,50,15,30
1753326523.7612615,50,15,30
1753326527.071018,50,15,30
1753326527.734704,50,15,30
1753326529.9887965,50,15,30
1753326533.9202185,50,15,30
1753326713.2606113,50,15,30
1753326734.9530346,50,15,30
1753326738.7199006,50,16,32
1753326741.5109105,50,22,44
1753326742.088726,50,25,50
1753326742.546171,50,26,52
1753326743.2469406,50,29,58
1753326743.465609,50,29,58
1753326743.4941173,50,29,58
1753326744.0394442,50,29,58
1753326744.6856656,50,29,58
1753326744.9265063,50,29,58
1753326744.9701695,50,29,58
1753326745.1508162,50,29,58
1753326745.790323,50,29,58
1753326745.8587046,50,29,58
1753326828.9836304,50,30,60
1753326929.6511142,50,31,62
1753327006.1038847,50,31,62
1753327006.1408288,50,31,62
1753327006.4676008,50,31,62
1753327007.2011764,50,31,62
1753327007.4608064,50,31,62
1753327020.9600236,50,31,62
1753327021.5796638,50,31,62
1753327023.2839918,50,31,62
1753327031.694479,50,31,62
1753327032.9219773,50,31,62
1753327033.8209016,50,31,62
1753327034.173214,50,31,62
1753327034.4316087,50,31,62
1753327034.6175928,50,31,62
1753327035.2049224,50,31,62
1753327035.356722,50,31,62
1753327221.9193792,50,35,70
1753327223.3278415,50,36,72
1753327223.433662,50,36,72
1753327223.6200857,50,36,72
1753327224.6618943,50,36,72
1753327238.380621,50,37,74
1753327244.2798028,50,39,78
1753327245.2429843,50,39,78
1753327249.4674497,50,41,82
1753327250.6620226,50,43,86
1753327252.2900114,50,46,92
1753327252.5286217,50,46,92
1753327253.1058123,50,46,92
1753327253.7419033,50,46,92
1753327253.9524753,50,46,92
1753327257.1675098,50,47,94
1753327498.7836902,50,47,94
1753327498.7984817,50,47,94
1753327533.7207031,50,49,98
1753327534.9831812,50,49,98
1753327595.141149,50,48,96
1753327595.1641407,50,48,96
1753327625.9919422,50,48,96
1753327700.3878508,50,47,94
1753327724.0789227,50,47,94
1753327746.784118,50,47,94
1753327799.1849222,50,47,94
1753327823.7573862,50,47,94
1753327823.925436,50,47,94
1753327824.1671803,50,47,94
1753327854.24843,50,47,94
1753327885.8361046,50,47,94
1753327918.1379187,50,47,94
1753327949.5856693,50,47,94
1753327980.7192972,50,47,94
1753328009.7750344,50,47,94
1753328041.0435169,50,47,94
1753328072.60168,50,47,94
1753328104.0826786,50,47,94
1753328135.7891245,50,47,94
1753328167.7441466,50,47,94
1753328199.3498523,50,47,94
1753328229.669645,50,47,94
1753328260.717367,50,47,94
1753328293.1345193,50,47,94
1753328325.2709055,50,47,94
1753328357.5886004,50,47,94
1753328388.582044,50,47,94
1753328419.749197,50,47,94
1753328449.7855444,50,47,94
1753328481.5754435,50,47,94
1753328514.3288975,50,47,94
1753328546.3473723,50,47,94
1753328578.4223535,50,47,94
1753328609.7255275,50,47,94
1753328642.1436746,50,47,94
1753328673.1805842,50,47,94
1753328704.5448203,50,47,94
1753328736.7319703,50,47,94
1753328767.8418565,50,47,94
1753328799.0288084,50,47,94
1753328830.1956031,50,47,94
1753328861.6035461,50,47,94
1753328891.1356726,50,47,94
1753328922.6655526,50,47,94
1753328954.16858,50,47,94
1753328986.204803,50,47,94
1753329017.7456975,50,47,94
1753329049.0868046,50,47,94
1753329080.680439,50,47,94
1753329111.8192563,50,47,94
1753329144.0000432,50,47,94
1753329176.8094206,50,47,94
1753329207.8985493,50,47,94
1753329240.2691326,50,47,94
1753329270.30794,50,47,94
1753329301.9002526,50,47,94
1753329333.1590827,50,47,94
1753329364.7187645,50,47,94
1753329395.8989577,50,47,94
1753329428.586876,50,47,94
1753329460.288197,50,47,94
1753329491.4908824,50,47,94
1753329523.5207891,50,47,94
1753329555.1777313,50,47,94
1753329587.2180889,50,47,94
1753329619.396956,50,47,94
1753329651.5592191,50,47,94
1753329683.8521733,50,47,94
1753329714.160999,50,47,94
1753329745.4526374,50,47,94
1753329777.3076782,50,47,94
1753329809.5543587,50,47,94
1753329840.1702611,50,46,92
1753329863.4491804,50,46,92
1753329920.4003146,50,36,72
1753329945.753449,50,31,62
1753329978.4075859,50,31,62
1753330010.7572432,50,31,62
1753330042.234001,50,31,62
1753330074.3991663,50,31,62
1753330108.2018564,50,31,62
1753330140.8905528,50,31,62
1753330172.9720519,50,31,62
1753330203.6531706,50,31,62
1753330233.4691982,50,30,60
1753330255.2480175,50,30,60
1753330307.3456285,50,30,60
1753330330.0480626,50,30,60
1753330361.201334,50,30,60
1753330390.8274908,50,29,58
1753330422.0288858,50,18,36
1753330453.9499726,50,17,34
1753330485.0294266,50,17,34
1753330515.578768,50,17,34
1753330547.7904263,50,17,34
1753330580.1453884,50,16,32
1753330615.7346547,50,16,32
1753330647.942527,50,16,32
1753330679.5987244,50,16,32
1753330710.7280009,50,16,32
1753330742.2379708,50,16,32
1753330773.9349024,50,16,32
1753330805.1025665,50,16,32
1753330836.1200447,50,16,32
1753330865.9162343,50,12,24
1753330899.0689642,50,2,4
1753330930.2296073,50,2,4
1753330963.0148365,50,2,4
1753330992.108361,50,2,4
1753331023.7953994,50,2,4
1753331053.646687,50,2,4
1753331086.9274707,50,2,4
1753331125.338596,50,2,4
1753331183.5276504,50,0,0
1753331183.7223575,50,0,0
1753331183.7283735,50,0,0
1753331183.878145,50,0,0
1753331183.9179375,50,0,0
1753331183.9452457,50,0,0
1753331184.0714872,50,0,0
1753331184.0936456,50,0,0
1753331184.193534,50,0,0
1753331184.3224313,50,0,0
1753331184.389657,50,0,0
1753331184.4622452,50,0,0
1753331184.5216167,50,0,0
1753331184.5555422,50,0,0
1753331184.696338,50,0,0
1753331455.7928596,50,0,0
1753331476.7243738,50,0,0
1753331507.788967,50,0,0
1753331538.5854774,50,0,0
1753331569.8937008,50,0,0
1753331599.2674615,50,0,0
1753331632.741405,50,0,0
1753331661.8746648,50,0,0
1753331691.654282,50,0,0
1753331718.7160158,50,0,0
1753331747.99297,50,0,0
1753331787.776429,50,0,0
1753331787.8108041,50,0,0
1753331842.9653058,50,0,0
1753332577.3426046,50,0,0
1753332616.408882,50,0,0
1753332653.5443811,50,0,0
1753332693.874027,50,0,0
1753332734.8354542,50,0,0
1753332773.9707944,50,0,0
1753332815.089231,50,0,0
1753332854.3079321,50,0,0
1753332871.8954496,50,0,0
1753332911.7554498,50,0,0
1753332930.8969028,50,0,0
1753332973.7283993,50,0,0
1753333009.794555,50,0,0
1753333047.3671834,50,0,0
1753333092.1040924,50,0,0
1753333131.0727968,50,0,0
1753333168.243565,50,0,0
1753333205.5801158,50,0,0
1753333246.0460353,50,0,0
1753333283.0919435,50,0,0
1753333302.140266,50,0,0
1753333340.5862403,50,0,0
1753333380.0946012,50,0,0
1753333418.2885404,50,0,0
1753333464.0505927,50,0,0
1753333501.4438195,50,0,0
1753333543.3013482,50,0,0
1753333582.616601,50,0,0
1753333624.8655634,50,0,0
1753333666.4600458,50,0,0
1753333707.919,50,0,0
1753333725.7667594,50,0,0
1753333765.1486602,50,0,0
1753333804.0450346,50,0,0
1753333823.8132513,50,0,0
1753333861.1728334,50,0,0
1753333877.881594,50,0,0
1753333908.7963798,50,0,0
1753333941.891052,50,0,0
1753333979.2217195,50,0,0
1753334010.8954055,50,0,0
1753334045.5033464,50,0,0
1753334078.4240246,50,0,0
1753334112.7734375,50,0,0
1753334145.9448743,50,0,0
1753334179.7309709,50,0,0
1753334211.5535703,50,0,0
1753334228.0847328,50,0,0
1753334260.811022,50,0,0
1753334293.4608717,50,0,0
1753334418.2466056,50,0,0
1753334451.533546,50,0,0
1753334484.9296124,50,0,0
1753334518.7124708,50,0,0
1753334549.9689496,50,0,0
1753334584.0449026,50,0,0
1753334615.1562197,50,0,0
1753334649.488923,50,0,0
1753334709.5133495,50,0,0
1753334741.0871735,50,0,0
1753334757.737533,50,0,0
1753334775.9251904,50,0,0
1753334793.6219044,50,0,0
1753334811.0624118,50,0,0
1753334829.5121894,50,0,0
1753334848.4658577,50,0,0
1753334866.4426324,50,0,0
1753334884.428957,50,0,0
1753334902.8117845,50,0,0
1753334921.4800458,50,0,0
1753334940.3260636,50,0,0
1753334957.7806,50,0,0
1753334977.9900434,50,0,0
1753334996.7329643,50,0,0
1753335015.0934637,50,0,0
1753335032.2336307,50,0,0
1753335051.4705985,50,0,0
1753335068.9158697,50,0,0
1753335086.407654,50,0,0
1753335103.864395,50,0,0
1753335120.7484212,50,0,0
1753335138.8006504,50,0,0
1753335156.0963957,50,0,0
1753335172.978133,50,0,0
1753335192.177002,50,0,0
1753335209.9267027,50,0,0
1753335226.768273,50,0,0
1753335244.3636844,50,0,0
1753335263.2082837,50,0,0
1753335280.3403306,50,0,0
1753335298.1660433,50,0,0
1753335316.3991637,50,0,0
1753335335.267216,50,0,0
1753335355.5773401,50,0,0
1753335373.9636872,50,0,0
1753335391.7895236,50,0,0
1753335410.4861648,50,0,0
1753335429.312018,50,0,0
1753335445.9490395,50,0,0
1753335464.9055045,50,0,0
1753335484.6487408,50,0,0
1753335502.0427241,50,0,0
1753335519.7476835,50,0,0
1753335539.155642,50,0,0
1753335558.334885,50,0,0
1753335575.8344195,50,0,0
1753335595.040605,50,0,0
1753335612.537396,50,0,0
1753335632.2654855,50,0,0
1753335650.4239373,50,0,0
1753335670.298157,50,0,0
1753335670.3256047,50,0,0
1753335670.3413575,50,0,0
1753335670.3424387,50,0,0
1753335670.3481603,50,0,0
1753335670.3585799,50,0,0
1753335670.3686001,50,0,0
1753335670.3751752,50,0,0
1753335670.3756058,50,0,0
1753335670.4086192,50,0,0
1753335670.4224865,50,0,0
1753335670.4256842,50,0,0
1753335670.4261184,50,0,0
1753335670.4819906,50,0,0
1753335671.3312113,50,0,0
1753335671.3329604,50,0,0
1753335859.0558677,50,1,2
1753335860.4327135,50,6,12
1753335860.8508039,50,6,12
1753335860.881677,50,6,12
1753335860.9307227,50,6,12
1753335861.5140471,50,9,18
1753335863.2793424,50,14,28
1753335863.5004473,50,16,32
1753335863.5968652,50,16,32
1753335863.6833751,50,16,32
1753335864.5148494,50,16,32
1753335864.641684,50,16,32
1753335865.80866,50,16,32
1753335866.0083344,50,16,32
1753335866.232075,50,16,32
1753335866.2929804,50,16,32
1753336036.5350347,50,16,32
1753336036.66048,50,16,32
1753336086.122683,50,18,36
1753336086.1418972,50,18,36
1753336087.5352685,50,18,36
1753336087.8715029,50,18,36
1753336088.0974476,50,18,36
1753336088.6314337,50,18,36
1753336088.685816,50,18,36
1753336088.6993182,50,18,36
1753336089.4770792,50,18,36
1753336093.1904235,50,18,36
1753336094.2635872,50,18,36
1753336094.5666933,50,18,36
1753336094.7582927,50,18,36
1753336095.012861,50,18,36
1753336097.1810133,50,18,36
1753336097.7460935,50,18,36
1753336258.9841938,50,19,38
1753336274.7664506,50,20,40
1753336277.648305,50,29,58
1753336277.711161,50,29,58
1753336277.7812345,50,29,58
1753336278.4493568,50,32,64
1753336278.5553997,50,32,64
1753336279.5217173,50,31,62
1753336279.9744015,50,32,64
1753336280.202322,50,32,64
1753336280.5373735,50,32,64
1753336280.7913096,50,32,64
1753336280.8289351,50,32,64
1753336281.0467935,50,32,64
1753336286.6899881,50,32,64
1753336290.7239885,50,32,64
1753336316.1795235,50,32,64
1753336471.059039,50,32,64
1753336489.726184,50,35,70
1753336490.0748224,50,35,70
1753336491.6910634,50,35,70
1753336502.05681,50,35,70
1753336502.409842,50,35,70
1753336502.9415922,50,35,70
1753336509.175126,50,35,70
1753336512.5846682,50,35,70
1753336518.879471,50,35,70
1753336518.919125,50,35,70
1753336518.9638162,50,35,70
1753336519.0032961,50,35,70
1753336520.7388062,50,35,70
1753336520.828261,50,35,70
1753336520.8834398,50,35,70
1753336546.7259743,50,36,72
1753336691.7243872,50,38,76
1753336693.1382942,50,38,76
1753336693.362121,50,38,76
1753336695.0151508,50,39,78
1753336695.274226,50,39,78
1753336697.949412,50,40,80
1753336702.124589,50,41,82
1753336705.6813724,50,43,86
1753336705.702597,50,43,86
1753336707.7083762,50,45,90
1753336708.6991389,50,45,90
1753336709.9638927,50,48,96
1753336711.0568311,50,48,96
1753336711.1211538,50,48,96
1753336867.6918805,50,50,100
1753336868.4889126,50,50,100
1753336939.9731965,50,50,100
1753336940.1314619,50,50,100
1753336940.3328452,50,50,100
1753336966.105231,50,50,100
1753336993.0249524,50,50,100
1753337020.1160524,50,50,100
1753337047.2800305,50,50,100
1753337075.0251248,50,50,100
1753337102.888667,50,50,100
1753337128.6454575,50,50,100
1753337154.268344,50,50,100
1753337181.026992,50,50,100
1753337207.6251607,50,50,100
1753337233.1084383,50,50,100
1753337258.558497,50,50,100
1753337283.3343427,50,50,100
1753337309.3377805,50,50,100
1753337334.2742617,50,50,100
1753337361.3703184,50,50,100
1753337386.890843,50,50,100
1753337413.0317397,50,50,100
1753337439.4906983,50,50,100
1753337464.2787857,50,50,100
1753337492.5795279,50,50,100
1753337519.035873,50,50,100
1753337544.6850553,50,50,100
1753337571.9833887,50,50,100
1753337598.2182446,50,50,100
1753337624.7939916,50,50,100
1753337651.8385746,50,50,100
1753337677.4833434,50,50,100
1753337704.9486074,50,50,100
1753337731.8158545,50,50,100
1753337757.774927,50,50,100
1753337784.837607,50,50,100
1753337811.6323338,50,50,100
1753337839.2720551,50,50,100
1753337870.8303416,50,50,100
1753337902.976849,50,50,100
1753337934.6505506,50,50,100
1753337966.6858757,50,50,100
1753337998.2667353,50,50,100
1753338030.3851368,50,50,100
1753338062.1887116,50,50,100
1753338093.7947419,50,50,100
1753338125.8816543,50,50,100
1753338158.8108559,50,50,100
1753338190.1640694,50,50,100
1753338222.7244825,50,50,100
1753338254.326528,50,50,100
1753338286.7287095,50,50,100
1753338318.885527,50,50,100
1753338350.3398573,50,50,100
1753338383.9111397,50,50,100
1753338415.866388,50,50,100
1753338447.319541,50,50,100
1753338478.4957423,50,50,100
1753338509.9971297,50,50,100
1753338541.5419269,50,50,100
1753338572.9141471,50,50,100
1753338604.0038962,50,50,100
1753338636.748464,50,50,100
1753338668.3987176,50,50,100
1753338700.250199,50,50,100
1753338732.0768893,50,50,100
1753338763.6393237,50,50,100
1753338795.1452007,50,50,100
1753338827.0216472,50,50,100
1753338859.115988,50,50,100
1753338891.3078856,50,50,100
1753338924.0702746,50,50,100
1753338955.8670099,50,50,100
1753338988.3504179,50,50,100
1753339020.9913104,50,50,100
1753339054.0215423,50,50,100
1753339087.1192207,50,50,100
1753339119.2506998,50,50,100
1753339152.1758444,50,50,100
1753339185.0575595,50,50,100
1753339218.013113,50,50,100
1753339252.162411,50,50,100
1753339283.7368548,50,50,100
1753339315.557114,50,50,100
1753339347.1384487,50,50,100
1753339379.3350573,50,50,100
1753339411.3741958,50,50,100
1753339443.0166535,50,50,100
1753339474.5770183,50,49,98
1753339507.0479403,50,39,78
1753339539.6879058,50,34,68
1753339571.798553,50,34,68
1753339605.9130635,50,34,68
1753339637.6785722,50,34,68
1753339668.4031184,50,34,68
1753339699.6662374,50,34,68
1753339731.8451328,50,34,68
1753339766.16864,50,32,64
1753339797.9735038,50,32,64
1753339829.4596913,50,32,64
1753339864.28879,50,32,64
1753339897.5968442,50,31,62
1753339942.3805394,50,21,42
1753339942.5619283,50,21,42
1753339942.5783925,50,21,42
1753339942.7460432,50,21,42
1753339942.985998,50,21,42
1753339943.0376449,50,21,42
1753339943.1741734,50,21,42
1753339943.2737164,50,21,42
1753339943.3479402,50,21,42
1753339943.4007103,50,21,42
1753339943.5362334,50,21,42
1753339943.556442,50,21,42
1753339943.5630765,50,21,42
1753339943.7176094,50,21,42
1753339943.740655,50,21,42
1753339943.834578,50,21,42
1753339943.9403067,50,21,42
1753339943.9465613,50,21,42
1753339944.123634,50,21,42
1753339944.430236,50,21,42
1753339944.4803069,50,21,42
1753339944.5180912,50,21,42
1753339944.87857,50,21,42
1753339945.2780247,50,21,42
1753339945.505573,50,21,42
1753339945.6789997,50,21,42
1753339945.6808202,50,21,42
1753339946.523812,50,21,42
1753339946.6987162,50,21,42
1753340499.0668585,50,2,4
1753341574.4070928,50,0,0
1753341627.5296237,50,0,0
1753341678.2415771,50,0,0
1753341729.5136364,50,0,0
1753341780.7288966,50,0,0
1753341864.4551294,50,0,0
1753341915.0985296,50,0,0
1753341938.2851694,50,0,0
1753341990.2941282,50,0,0
1753342040.158357,50,0,0
1753342092.8116782,50,0,0
1753342116.3956974,50,0,0
1753342166.8750489,50,0,0
1753342190.3491197,50,0,0
1753342241.2259889,50,0,0
1753342291.8917584,50,0,0
1753342315.7803948,50,0,0
1753342366.8788085,50,0,0
1753342417.8992293,50,0,0
1753342468.7736776,50,0,0
1753342519.0695832,50,0,0
1753342568.9916518,50,0,0
1753342619.7809582,50,0,0
1753342670.5653667,50,0,0
1753342721.3526218,50,0,0
1753342772.793525,50,0,0
1753342824.468081,50,0,0
1753342875.042037,50,0,0
1753342925.6038513,50,0,0
1753342977.0313911,50,0,0
1753343027.1530359,50,0,0
1753343077.2427785,50,0,0
1753343128.5472252,50,0,0
1753343179.3006043,50,0,0
1753343230.3037632,50,0,0
1753343281.8258154,50,0,0
1753343335.0087144,50,0,0
1753343387.1569712,50,0,0
1753343438.1620514,50,0,0
1753343490.2586555,50,0,0
1753343543.423127,50,0,0
1753343568.6914055,50,0,0
1753343592.5541935,50,0,0
1753343643.8292868,50,0,0
1753343695.841107,50,0,0
1753343748.6826267,50,0,0
1753343800.610261,50,0,0
1753343851.6155984,50,0,0
1753343901.908722,50,0,0
1753343952.7660172,50,0,0
1753344144.2986135,50,0,0
1753344195.420315,50,0,0
1753344247.2058425,50,0,0
1753344299.2365377,50,0,0
1753344349.4348547,50,0,0
1753344402.0200586,50,0,0
1753344473.298056,50,0,0
1753344523.891841,50,0,0
1753344546.890973,50,0,0
1753344574.2096565,50,0,0
1753344601.4223013,50,0,0
1753344628.8895457,50,0,0
1753344656.395494,50,0,0
1753344684.6156244,50,0,0
1753344711.9878411,50,0,0
1753344739.875969,50,0,0
1753344767.0582383,50,0,0
1753344794.4921536,50,0,0
1753344821.934588,50,0,0
1753344850.5979846,50,0,0
1753344877.9028006,50,0,0
1753344906.0394359,50,0,0
1753344933.4217696,50,0,0
1753344961.9446027,50,0,0
1753344989.3429418,50,0,0
1753345016.737683,50,0,0
1753345043.6928573,50,0,0
1753345070.814818,50,0,0
1753345098.7646532,50,0,0
1753345125.8624635,50,0,0
1753345154.8378327,50,0,0
1753345182.1294332,50,0,0
1753345209.6618078,50,0,0
1753345236.8642364,50,0,0
1753345264.2023299,50,0,0
1753345292.116735,50,0,0
1753345319.6950781,50,0,0
1753345347.6045954,50,0,0
1753345374.8576896,50,0,0
1753345402.4887006,50,0,0
1753345429.6438704,50,0,0
1753345457.7600617,50,0,0
1753345485.038031,50,0,0
1753345512.7430344,50,0,0
1753345540.4236612,50,0,0
1753345567.6133306,50,0,0
1753345594.8958797,50,0,0
1753345622.2321162,50,0,0
1753345648.9104831,50,0,0
1753345677.1285555,50,0,0
1753345704.7640197,50,0,0
1753345744.3152823,50,0,0
1753345773.2466052,50,0,0
1753345800.9146628,50,0,0
1753345828.9834116,50,0,0
1753345857.9270287,50,0,0
1753345887.044383,50,0,0
1753345914.728838,50,0,0
1753345943.9393601,50,0,0
1753345944.2258446,50,0,0
1753345944.2591105,50,0,0
1753345944.2872427,50,0,0
1753345944.3848155,50,0,0
1753345944.557837,50,0,0
1753345944.5585494,50,0,0
1753345944.5592587,50,0,0
1753345944.580854,50,0,0
1753345944.6248095,50,0,0
1753345944.8452885,50,0,0
1753345944.8461075,50,0,0
1753345944.915163,50,0,0
1753345944.915966,50,0,0
1753345944.9888315,50,0,0
1753345945.937711,50,0,0
1753346224.8123589,50,2,4
1753346225.8557148,50,2,4
1753346227.479416,50,5,10
1753346228.1064663,50,6,12
1753346228.6060421,50,6,12
1753346230.305777,50,9,18
1753346232.4483478,50,16,32
1753346232.6194248,50,16,32
1753346233.682343,50,16,32
1753346234.0734286,50,16,32
1753346234.2237165,50,16,32
1753346234.4737608,50,16,32
1753346234.5775318,50,16,32
1753346234.6792815,50,16,32
1753346234.9703145,50,16,32
1753346235.0205345,50,16,32
1753346548.2669926,50,16,32
1753346561.9703941,50,16,32
1753346567.4402707,50,16,32
1753346575.8657842,50,16,32
1753346577.414316,50,16,32
1753346580.8322175,50,16,32
1753346585.9634886,50,16,32
1753346588.0623577,50,16,32
1753346594.881652,50,16,32
1753346602.2519863,50,16,32
1753346602.2820196,50,16,32
1753346604.534971,50,16,32
1753346604.8943245,50,16,32
1753346605.0855942,50,16,32
1753346605.5613241,50,16,32
1753346724.4320784,50,16,32
1753346800.1531253,50,17,34
1753346845.331171,50,18,36
1753346864.500081,50,19,38
1753346876.1998541,50,21,42
1753346878.1332943,50,21,42
1753346881.4205964,50,23,46
1753346883.0033302,50,24,48
1753346884.3050172,50,24,48
1753346897.8556173,50,25,50
1753346903.3429184,50,26,52
1753346907.7409651,50,31,62
1753346907.9657884,50,31,62
1753346908.253026,50,31,62
1753346908.9203217,50,31,62
1753346909.6222026,50,31,62
1753347029.153783,50,32,64
1753347099.963284,50,32,64
1753347196.7550945,50,32,64
1753347225.759216,50,32,64
1753347230.1346097,50,32,64
1753347233.5330725,50,32,64
1753347279.079496,50,32,64
1753347279.6030712,50,32,64
1753347287.8164601,50,32,64
1753347287.9311364,50,32,64
1753347288.0840166,50,32,64
1753347288.2675252,50,32,64
1753347288.4161396,50,32,64
1753347288.4727452,50,32,64
1753347289.0504053,50,32,64
1753347289.440446,50,32,64
1753347309.5751402,50,32,64
1753347325.5948462,50,33,66
1753347392.8600914,50,34,68
1753347450.124257,50,35,70
1753347453.1756046,50,36,72
1753347466.90711,50,37,74
1753347520.0778131,50,38,76
1753347620.4098651,50,40,80
1753347620.4331594,50,40,80
1753347623.1651602,50,43,86
1753347624.6926107,50,43,86
1753347624.9029377,50,43,86
1753347630.5635467,50,48,96
1753347631.7731488,50,48,96
1753347632.042515,50,48,96
1753347632.1355867,50,48,96
1753347632.5906441,50,48,96
1753347652.8845413,50,48,96
1753347707.197022,50,48,96
1753347937.0367799,50,50,100
1753347937.13336,50,50,100
1753348016.7159626,50,50,100
1753348018.0563738,50,50,100
1753348018.3657346,50,50,100
1753348054.1172485,50,50,100
1753348090.2568777,50,50,100
1753348125.886324,50,50,100
1753348162.4204113,50,50,100
1753348197.9205005,50,50,100
1753348233.304251,50,50,100
1753348270.1510797,50,50,100
1753348307.790838,50,50,100
1753348345.1892269,50,50,100
1753348381.5313296,50,50,100
1753348416.6743932,50,50,100
1753348453.1912353,50,50,100
1753348489.2345889,50,50,100
1753348524.5348065,50,50,100
1753348561.5851157,50,50,100
1753348592.9947362,50,49,98
1753348620.457638,50,49,98
1753348681.6861095,50,49,98
1753348707.8047454,50,49,98
1753348744.8389049,50,49,98
1753348795.0810328,50,49,98
1753348830.7802002,50,49,98
1753348865.22253,50,49,98
1753348899.452651,50,49,98
1753348932.6518972,50,49,98
1753348965.3207955,50,49,98
1753348996.6433554,50,49,98
1753349028.7170985,50,49,98
1753349060.8263626,50,49,98
1753349092.0625894,50,49,98
1753349123.0022492,50,49,98
1753349154.4231076,50,49,98
1753349185.940161,50,49,98
1753349217.699549,50,49,98
1753349250.2230582,50,49,98
1753349282.7634463,50,49,98
1753349315.2608953,50,49,98
1753349341.203096,50,48,96
1753349362.549545,50,48,96
1753349414.9106176,50,48,96
1753349437.9027007,50,48,96
1753349467.9079847,50,48,96
1753349497.1048415,50,48,96
1753349527.5551414,50,48,96
1753349556.0905504,50,48,96
1753349585.2846816,50,48,96
1753349613.5558534,50,48,96
1753349642.7339628,50,48,96
1753349671.139844,50,48,96
1753349700.2612045,50,48,96
1753349729.098679,50,48,96
1753349758.296153,50,48,96
1753349788.0873067,50,48,96
1753349816.7508008,50,48,96
1753349845.5835984,50,48,96
1753349874.9372163,50,38,76
1753349917.167876,50,38,76
1753349917.2265422,50,38,76
1753349917.274765,50,38,76
1753349917.320259,50,38,76
1753349917.3574405,50,38,76
1753349917.393031,50,38,76
1753349917.4084382,50,38,76
1753349917.4903328,50,38,76
1753349917.5237064,50,38,76
1753349917.8043873,50,38,76
1753350091.1697073,50,32,64
1753350110.7148306,50,32,64
1753350139.7601495,50,32,64
1753350167.3303988,50,32,64
1753350195.6441052,50,32,64
1753350225.533139,50,32,64
1753350254.333172,50,32,64
1753350282.823954,50,32,64
1753350310.889563,50,32,64
1753350339.0905871,50,32,64
1753350367.750951,50,31,62
1753350393.0154505,50,31,62
1753350414.465621,50,30,60
1753350459.5241115,50,30,60
1753350479.6221452,50,30,60
1753350504.1145742,50,29,58
1753350504.118185,50,29,58
1753350520.812321,50,29,58
1753350520.8292418,50,29,58
1753350520.842058,50,29,58
1753350520.848633,50,29,58
1753350520.8685668,50,29,58
1753350521.8701806,50,29,58
1753350571.5479813,50,29,58
1753350571.9557028,50,29,58

Copy raw data to clipboard

Download »worker_usage.csv« as file

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

timestamp,ram_usage_mb,cpu_usage_percent
1753270333,649.08203125,38.3
1753270343,649.08203125,38.1
1753270347,649.453125,38.3
1753270347,649.453125,47.4
1753270347,649.453125,39.0
1753270347,649.7109375,38.9
1753270347,649.7109375,47.4
1753271363,729.28125,39.4
1753271363,729.28125,45.0
1753271363,729.28125,43.6
1753271363,729.28125,45.0
1753271520,729.5078125,44.4
1753271572,730.0390625,44.4
1753272083,730.359375,41.2
1753272881,730.89453125,69.2
1753272881,730.89453125,42.1
1753272948,732.4609375,50.0
1753272948,732.4609375,26.7
1753273182,732.390625,47.1
1753273282,732.48046875,38.3
1753273282,732.48046875,42.1
1753273626,732.76171875,63.6
1753273626,732.76171875,42.1
1753274140,732.80078125,38.3
1753274140,732.80078125,42.1
1753274232,732.984375,38.3
1753274232,732.984375,42.1
1753274410,733.18359375,44.4
1753274500,738.0390625,63.6
1753274708,738.22265625,38.3
1753274708,738.22265625,42.9
1753277204,823.578125,55.6
1753277247,823.91015625,42.1
1753277247,823.91015625,55.0
1753277247,823.91015625,49.7
1753277563,823.98046875,44.7
1753277563,823.98046875,50.0
1753278067,824.109375,46.7
1753278067,824.109375,45.5
1753278188,824.6171875,52.6
1753278246,826.3984375,43.3
1753278246,826.3984375,52.4
1753278355,826.5,49.3
1753278355,826.5,52.6
1753278414,827.27734375,49.4
1753278414,827.27734375,52.6
1753278492,828.5390625,55.6
1753278492,828.5390625,76.9
1753278587,828.63671875,49.5
1753278587,828.63671875,58.3
1753278688,828.82421875,49.5
1753278688,828.82421875,50.0
1753278796,828.765625,55.6
1753278857,828.734375,49.5
1753278857,828.734375,52.6
1753278910,829.140625,47.8
1753278910,829.140625,52.6
1753280382,829.8359375,47.8
1753280382,829.8359375,40.0
1753283387,912.16015625,43.3
1753283387,912.16015625,28.6
1753283387,912.16015625,32.6
1753283387,912.16015625,43.8
1753284244,912.19921875,43.8
1753284383,912.35546875,37.5
1753284432,912.515625,31.2
1753284432,912.515625,35.3
1753284681,913.0,40.3
1753284681,913.0,31.6
1753284681,913.0,43.8
1753284682,913.25390625,50.0
1753284682,913.25390625,31.1
1753284758,913.71484375,31.4
1753284758,913.71484375,40.1
1753284758,913.71484375,37.5
1753284758,913.71484375,40.0
1753284830,916.9453125,38.9
1753284830,916.9453125,31.4
1753284830,916.9453125,38.9
1753284929,917.55859375,46.7
1753284982,917.48828125,31.2
1753284982,917.48828125,35.3
1753285146,917.625,37.5
1753285247,917.5859375,31.2
1753285247,917.5859375,35.3
1753285555,917.61328125,31.1
1753285555,917.61328125,37.5
1753285625,917.73828125,31.1
1753285625,917.73828125,35.3
1753285933,917.96484375,31.2
1753286352,918.578125,31.1
1753286352,918.578125,35.3
1753286689,919.09765625,31.1
1753291148,1000.66796875,52.4
1753291159,1000.66796875,51.2
1753291160,1000.66796875,43.5
1753291161,1000.66796875,52.4
1753291161,1000.66796875,51.3
1753291161,1000.66796875,50.7
1753291162,1000.66796875,50.9
1753291162,1000.66796875,43.2
1753291162,1000.66796875,50.6
1753291162,1000.66796875,50.6
1753291162,1000.66796875,58.8
1753291163,1000.66796875,50.8
1753291163,1000.66796875,43.2
1753291163,1000.66796875,50.7
1753291163,1000.66796875,50.4
1753291337,1005.25,50.6
1753291337,1005.25,50.0
1753291337,1005.25,43.1
1753291337,1005.25,58.6
1753291397,1005.4140625,41.3
1753291397,1005.4140625,50.0
1753291397,1005.4140625,50.0
1753291397,1005.4140625,54.5
1753291430,1005.4140625,50.5
1753291430,1005.4140625,52.6
1753291528,1005.4609375,48.4
1753291528,1005.4609375,45.5
1753291528,1005.4609375,50.3
1753291528,1005.4609375,47.8
1753291685,1005.41015625,50.2
1753291887,1005.44140625,55.6
1753291964,1005.73828125,50.3
1753292148,1005.6484375,73.3
1753292148,1005.6484375,52.6
1753292731,1005.83203125,50.2
1753292731,1005.83203125,57.1
1753292956,1005.96875,46.8
1753292956,1005.96875,47.6
1753293053,1006.28515625,50.1
1753293132,1006.2265625,50.1
1753293132,1006.2265625,52.6
1753293232,1006.51953125,50.1
1753293232,1006.51953125,52.6
1753293339,1008.84765625,50.1
1753293339,1008.84765625,50.1
1753293339,1008.84765625,45.5
1753293339,1008.84765625,47.6
1753293339,1008.84765625,50.2
1753293339,1008.84765625,50.6
1753293474,1009.18359375,50.1
1753293474,1009.18359375,50.0
1753293474,1009.18359375,50.2
1753293474,1009.18359375,45.5
1753299378,1066.546875,47.6
1753299378,1066.546875,36.8
1753299430,1067.23046875,48.2
1753299430,1067.23046875,32.7
1753301953,1066.44140625,43.5
1753301953,1066.44140625,35.3
1753302164,1066.765625,40.0
1753302396,1066.74609375,42.6
1753302396,1066.74609375,38.9
1753308673,1192.82421875,43.5
1753308673,1192.82421875,50.3
1753308677,1192.82421875,49.4
1753308677,1192.82421875,50.3
1753308787,1192.8671875,43.7
1753308788,1192.8671875,50.0
1753308858,1192.765625,41.1
1753308858,1192.765625,50.0
1753308859,1192.765625,49.0
1753308859,1192.765625,47.6
1753309974,1192.94140625,52.6
1753310713,1192.3515625,46.9
1753310713,1192.3515625,47.4
1753310912,1186.1171875,45.5
1753310912,1186.1171875,45.0
1753317359,1188.84375,40.2
1753317359,1188.84375,29.6
1753317359,1188.84375,26.5
1753317495,1189.984375,43.0
1753317495,1189.984375,33.3
1753319526,1189.7890625,40.1
1753319526,1189.7890625,29.4
1753319840,1193.19140625,25.3
1753319840,1193.19140625,29.4
1753327626,1243.15625,41.5
1753327626,1243.15625,37.5
1753327746,1243.125,52.9
1753327823,1246.92578125,37.6
1753327823,1246.92578125,47.4
1753327823,1246.92578125,43.6
1753327823,1246.92578125,47.4
1753329863,1248.65625,55.6
1753329863,1248.65625,30.8
1753330255,1251.0859375,39.4
1753336939,1313.49609375,34.7
1753336939,1313.49609375,25.0
1753336940,1313.49609375,29.2
1753336940,1313.49609375,33.3
1753348016,1412.64453125,47.0
1753348016,1412.64453125,52.4
1753348018,1412.64453125,48.2
1753348018,1412.64453125,50.0
1753348620,1412.76171875,41.5
1753348620,1412.76171875,42.1
1753349362,1414.5078125,40.1
1753349362,1414.5078125,23.8
1753350414,1423.02734375,43.7
1753350571,1429.0859375,40.0
1753350571,1429.0859375,22.7
1753350572,1429.0859375,39.4
1753350572,1429.0859375,26.7

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

Parameter statistics

Correlation Heatmap Explanation

This is a heatmap that visualizes the correlation between numerical columns in a dataset. The values represented in the heatmap show the strength and direction of relationships between different variables.

How It Works

The heatmap uses a matrix to represent correlations between each pair of numerical columns. The calculation behind this is based on the concept of "correlation," which measures how strongly two variables are related. A correlation can be positive, negative, or zero:

• Positive correlation: Both variables increase or decrease together (e.g., if the temperature rises, ice cream sales increase).
• Negative correlation: As one variable increases, the other decreases (e.g., as the price of a product rises, the demand for it decreases).
• Zero correlation: There is no relationship between the two variables (e.g., height and shoe size might show zero correlation in some contexts).

Color Scale: Yellow to Purple (Viridis)

The heatmap uses a color scale called "Viridis," which ranges from yellow to purple. Here's what the colors represent:

• Yellow (brightest): A strong positive correlation (close to +1). This indicates that as one variable increases, the other increases in a very predictable manner.
• Green: A moderate positive correlation. Variables are still positively related, but the relationship is not as strong.
• Blue: A weak or near-zero correlation. There is a small or no discernible relationship between the variables.
• Purple (darkest): A strong negative correlation (close to -1). This indicates that as one variable increases, the other decreases in a very predictable manner.

What the Heatmap Shows

In the heatmap, each cell represents the correlation between two numerical columns. The color of the cell is determined by the correlation coefficient: from yellow for strong positive correlations, through green and blue for weaker correlations, to purple for strong negative correlations.