OmniOpt2

Setting	Value
Model for non-random steps	BOTORCH_MODULAR
Max. nr. evaluations	3
Number random steps	2
Nr. of workers (parameter)	1
Main process memory (GB)	8
Worker memory (GB)	4
OO_Info_int_param	OO_Info_int_param_two	OO_Info_float_param	OO_Info_choice_param	int_param	float_param	int_param_two	choice_param	RESULT
-77	-79	-25.9017950296402	1	-77	-25.901795029640198	-79	1	-147005.329360539
Generation Node	Total	COMPLETED
SOBOL	2	2
BOTORCH_MODULAR	1	1
Name	Type	Lower bound	Upper bound	Values	Type	Log Scale?
int_param	range	-100	10		int	No
float_param	range	-100	10		float	No
choice_param	choice			1, 2, 4, 8, 16, hallo
int_param_two	range	-100	10		int	No
Failed	Succeeded	Running	Total
0	3	0	3
name	min/max
RESULT	min
Key	Value
config_yaml	None
config_toml	None
config_json	None
num_random_steps	2
max_eval	3
run_program	[['Li8udGVzdHMvb3B0aW1pemF0aW9uX2V4YW1wbGUgIC0taW50X3BhcmFtPSclKGludF9wYXJhbSknIC0tZmxvYXRfcGFyYW09JyUoZmxvYXRfcGFyYW0pJyAtLWNob2ljZV9wYXJhbT0nJShjaG9pY2VfcGFyYW0pJyAtLWludF9wYXJhbV90d289…
experiment_name	__main__tests__BOTORCH_MODULAR___local_nogridsearch
mem_gb	4
parameter	[['int_param', 'range', '-100', '10', 'int'], ['float_param', 'range', '-100', '10', 'float'], ['choice_param', 'choice', '1,2,4,8,16,hallo'], ['int_param_two', 'range', '-100', '10',
	'int']]
continue_previous_job	None
experiment_constraints	None
run_dir	runs
seed	None
verbose_tqdm	False
model	BOTORCH_MODULAR
gridsearch	False
occ	False
show_sixel_scatter	False
show_sixel_general	False
show_sixel_trial_index_result	False
follow	True
send_anonymized_usage_stats	True
ui_url	None
root_venv_dir	/home/s3811141
exclude	None
main_process_gb	8
max_nr_of_zero_results	10
abbreviate_job_names	False
orchestrator_file	None
checkout_to_latest_tested_version	False
live_share	True
disable_tqdm	False
disable_previous_job_constraint	False
workdir
occ_type	euclid
result_names	['RESULT=min']
minkowski_p	2
signed_weighted_euclidean_weights
generation_strategy	None
generate_all_jobs_at_once	True
revert_to_random_when_seemingly_exhausted	False
load_data_from_existing_jobs	[]
n_estimators_randomforest	100
max_attempts_for_generation	20
external_generator	None
username	None
max_failed_jobs	None
num_cpus_main_job	None
calculate_pareto_front_of_job	[]
show_generate_time_table	True
force_choice_for_ranges	False
max_abandoned_retrial	20
share_password	None
dryrun	False
db_url	None
run_program_once	None
worker_generator_path	None
save_to_database	False
range_max_difference	1000000
skip_search	False
dont_warm_start_refitting	False
refit_on_cv	False
fit_out_of_design	False
fit_abandoned	False
dont_jit_compile	False
num_restarts	20
raw_samples	1024
max_num_of_parallel_sruns	16
no_transform_inputs	False
no_normalize_y	False
transforms	[]
number_of_generators	1
num_parallel_jobs	1
worker_timeout	5
slurm_use_srun	False
time	60
partition	alpha
reservation	None
force_local_execution	True
slurm_signal_delay_s	0
nodes_per_job	1
cpus_per_task	1
account	None
gpus	1
dependency	None
run_mode	local
verbose	False
verbose_break_run_search_table	False
debug	False
flame_graph	False
memray	False
no_sleep	False
tests	False
show_worker_percentage_table_at_end	False
auto_exclude_defective_hosts	True
run_tests_that_fail_on_taurus	False
raise_in_eval	False
show_ram_every_n_seconds	0
show_generation_and_submission_sixel	False
just_return_defaults	False
prettyprint	False
runtime_debug	False
debug_stack_regex
debug_stack_trace_regex	None
show_func_name	False
beartype	False
Parameter	Min	Max	Mean	Std Dev	Count
run_time	6	7	6.3333	0.4714	3
OO_Info_int_param	-100	-13	-63.3333	36.8088	3
OO_Info_int_param_two	-79	-18	-39.3333	28.0753	3
OO_Info_float_param	-91.5594	10	-35.8204	42.0504	3
OO_Info_choice_param	1	8	3.3333	3.2998	3
RESULT	-147005.3294	-101959.6463	-122184.4986	18674.7718	3
int_param	-100	-13	-63.3333	36.8088	3
float_param	-91.5594	10	-35.8204	42.0504	3
int_param_two	-79	-18	-39.3333	28.0753	3
choice_param	1	8	3.3333	3.2998	3
submitit INFO (2025-10-24 16:24:17,440) - Starting with JobEnvironment(job_id=3272551, hostname=c1, local_rank=0(1), node=0(1), global_rank=0(1))
submitit INFO (2025-10-24 16:24:17,442) - Loading pickle: /data/horse/ws/s3811141-oo_crap/omniopt/runs/__main__tests__BOTORCH_MODULAR___local_nogridsearch/1/single_runs/3272551/3272551_submitted.pkl
Trial-Index: 0
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.

Parameters: {"int_param": -77, "float_param": -25.901795029640198, "int_param_two": -79, "choice_param": 1}
Debug-Infos: 
========
DEBUG INFOS START:
Program-Code: ./.tests/optimization_example  --int_param='-77' --float_param='-25.90179502964019775391' --choice_param='1' --int_param_two='-79' --nr_results=1
pwd: /data/horse/ws/s3811141-oo_crap/omniopt
File: ./.tests/optimization_example
UID: 2105408
GID: 200270
Status-Change-Time: 1755346813.0
Size: 5714 Bytes
Permissions: -rwxr-xr-x
Owner: s3811141
Last access: 1761315860.0
Last modification: 1755343211.0
Hostname: c1
========
DEBUG INFOS END

./.tests/optimization_example  --int_param='-77' --float_param='-25.90179502964019775391' --choice_param='1' --int_param_two='-79' --nr_results=1
stdout:
 OO-Info: int_param: -77
OO-Info: int_param_two: -79
OO-Info: float_param: -25.90179502964019775391
OO-Info: choice_param: 1
RESULT: -147005.329360539

stderr was empty
Result: {'RESULT': -147005.329360539}
Final-results: {'RESULT': -147005.329360539}
execute_nvidia_smi: An error occurred: Command '['nvidia-smi', 
'--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero 
exit status 255.
EXIT_CODE: 0
submitit INFO (2025-10-24 16:24:26,986) - Job completed successfully
submitit INFO (2025-10-24 16:24:26,988) - Exiting after successful completion
Experiment overview

Best RESULT, min (total: 3)

Job Summary per Generation Node

Experiment parameters

Number of evaluations

Result names and types

Last progressbar status

Git-Version

Arguments Overview

`RESULT` (goal: minimize)

Parameter statistics

Experiment overview

Best RESULT, min (total: 3)

Job Summary per Generation Node

Experiment parameters

Number of evaluations

Result names and types

Last progressbar status

Git-Version

Arguments Overview

RESULT (goal: minimize)

Parameter statistics

`RESULT` (goal: minimize)
