GUI page with all the settings of this job

Experiment overview

Best RESULT, min (total: 3)

Job Summary per Generation Node

Experiment parameters

Number of evaluations

Result names and types

Last progressbar status

Git-Version

Copy raw data to clipboard

Download »results.csv« as file

trial_index,submit_time,queue_time,worker_generator_uuid,start_time,end_time,run_time,program_string,exit_code,signal,hostname,OO_Info_int_param,OO_Info_int_param_two,OO_Info_float_param,OO_Info_choice_param,arm_name,trial_status,generation_node,RESULT,int_param,float_param,int_param_two,choice_param
0,1766938863,3,85311758-5615-4dd9-b831-12ec6f75bf09,1766938866,1766938872,6,./.tests/optimization_example  --int_param='-56' --float_param='-99.68751351349055767059' --choice_param='1' --int_param_two='-2' --nr_results=1,0,,c1,-56,-2,-99.68751351349055767059326171875,1,0_0,COMPLETED,SOBOL,-147740.47485637300997041165828704834,-56,-99.68751351349055767059326171875,-2,1
1,1766938877,2,85311758-5615-4dd9-b831-12ec6f75bf09,1766938879,1766938885,6,./.tests/optimization_example  --int_param='-34' --float_param='9.02868384495377540588' --choice_param='8' --int_param_two='-77' --nr_results=1,0,,c1,-34,-77,9.0286838449537754058837890625,8,1_0,COMPLETED,SOBOL,-82209.622935225896071642637252807617,-34,9.0286838449537754058837890625,-77,8
2,1766938913,3,85311758-5615-4dd9-b831-12ec6f75bf09,1766938916,1766938922,6,./.tests/optimization_example  --int_param='10' --float_param='-100' --choice_param='1' --int_param_two='10' --nr_results=1,0,,c1,10,10,-100,1,2_0,COMPLETED,BOTORCH_MODULAR,-111193.320000000006984919309616088867,10,-100,10,1

Copy raw data to clipboard

Download »results.csv« as file

Copy raw data to clipboard

Download »oo_errors.txt« as file

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

Copy raw data to clipboard

Download »oo_errors.txt« as file

Copy raw data to clipboard

Download »progressbar« as file

2025-12-28 17:21:02 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, Started OmniOpt2 run...
2025-12-28 17:21:03 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, getting new HP set (no sbatch)
2025-12-28 17:21:03 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, requested 1 jobs, got 1, 0.22 s/job
2025-12-28 17:21:03 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, eval #1/1 start                    
2025-12-28 17:21:03 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, starting new job                   
2025-12-28 17:21:03 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, started new job                    
2025-12-28 17:21:13 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, new result: RESULT: -147740.474856 
2025-12-28 17:21:15 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, finishing jobs, finished 1 job
2025-12-28 17:21:15 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, getting new HP set (no sbatch)
2025-12-28 17:21:16 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, requested 1 jobs, got 1, 0.09 s/job
2025-12-28 17:21:17 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, eval #1/1 start                    
2025-12-28 17:21:17 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, starting new job                   
2025-12-28 17:21:17 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, started new job                    
2025-12-28 17:21:27 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, new result: RESULT: -82209.622935  
2025-12-28 17:21:30 (85311758-5615-4dd9-b831-12ec6f75bf09): SOBOL, best RESULT: -147740.474856373, finishing jobs, finished 1 job     
2025-12-28 17:21:53 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, getting new HP set (no sbatch)
2025-12-28 17:21:53 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, requested 1 jobs, got 1, 22.91 s/job
2025-12-28 17:21:53 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, eval #1/1 start                     
2025-12-28 17:21:53 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, starting new job                    
2025-12-28 17:21:53 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, started new job                     
2025-12-28 17:22:04 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, new result: RESULT: -111193.320000  
2025-12-28 17:22:06 (85311758-5615-4dd9-b831-12ec6f75bf09): BOTORCH_MODULAR, best RESULT: -147740.474856373, finishing jobs, finished 1 job      

Copy raw data to clipboard

Download »progressbar« as file

Copy raw data to clipboard

Download »job_submit_durations.txt« as file

         Job submission durations          
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃   Batch ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│       1 │   0.245 │    1 │        0.245 │
│       2 │   0.211 │    1 │        0.211 │
│       3 │   0.209 │    1 │        0.209 │
├─────────┼─────────┼──────┼──────────────┤
│ Average │   0.222 │      │              │
│  Median │   0.211 │      │              │
│   Total │   0.666 │      │              │
│     Max │   0.245 │      │              │
│     Min │   0.209 │      │              │
└─────────┴─────────┴──────┴──────────────┘

Copy raw data to clipboard

Download »job_submit_durations.txt« as file

Copy raw data to clipboard

Download »generation_times.txt« as file

           Model generation times            
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃ Iteration ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│         1 │   0.219 │    1 │        0.219 │
│         2 │   0.092 │    1 │        0.092 │
│         3 │  22.906 │    1 │       22.906 │
├───────────┼─────────┼──────┼──────────────┤
│   Average │   7.739 │      │              │
│    Median │   0.219 │      │              │
│     Total │  23.218 │      │              │
│       Max │  22.906 │      │              │
│       Min │   0.092 │      │              │
└───────────┴─────────┴──────┴──────────────┘

Copy raw data to clipboard

Download »generation_times.txt« as file

Arguments Overview

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

timestamp,ram_usage_mb,cpu_usage_percent
1766938859,798.1328125,15.1

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

RESULT (goal: minimize)

Best value: -147740.47485637
Achieved at:
- run_time = 6
- OO_Info_int_param = -56
- OO_Info_int_param_two = -2
- OO_Info_float_param = -99.687513513491
- OO_Info_choice_param = 1
- int_param = -56
- float_param = -99.687513513491
- int_param_two = -2
- choice_param = 1

Parameter statistics