GUI page with all the settings of this job

Experiment overview

Best RESULT, min (total: 3)

Job Summary per Generation Node

Experiment parameters

Number of evaluations

Result names and types

Last progressbar status

Git-Version

Copy raw data to clipboard

Download »results.csv« as file

trial_index,submit_time,queue_time,worker_generator_uuid,start_time,end_time,run_time,program_string,exit_code,signal,hostname,OO_Info_int_param,OO_Info_int_param_two,OO_Info_float_param,OO_Info_choice_param,arm_name,trial_status,generation_node,RESULT,int_param,float_param,int_param_two,choice_param
0,1767003485,3,57454b0b-6448-45a1-bb13-28267c474e30,1767003488,1767003494,6,./.tests/optimization_example  --int_param='-25' --float_param='-94.53740235418081283569' --choice_param='8' --int_param_two='-49' --nr_results=1,0,,c1,-25,-49,-94.537402354180798624838644172996,8,0_0,COMPLETED,SOBOL,-137118.664674180996371433138847351074,-25,-94.537402354180812835693359375,-49,8
1,1767003501,3,57454b0b-6448-45a1-bb13-28267c474e30,1767003504,1767003510,6,./.tests/optimization_example  --int_param='-65' --float_param='-5.83621641620993614197' --choice_param='2' --int_param_two='-4' --nr_results=1,0,,c1,-65,-4,-5.8362164162099361419677734375,2,1_0,COMPLETED,SOBOL,-51367.155862189400068018585443496704,-65,-5.8362164162099361419677734375,-4,2
2,1767003539,2,57454b0b-6448-45a1-bb13-28267c474e30,1767003541,1767003548,7,./.tests/optimization_example  --int_param='-76' --float_param='-100' --choice_param='8' --int_param_two='-100' --nr_results=1,0,,c1,-76,-100,-100,8,2_0,COMPLETED,BOTORCH_MODULAR,-285542.919999999983701854944229125977,-76,-100,-100,8

Copy raw data to clipboard

Download »results.csv« as file

Copy raw data to clipboard

Download »oo_errors.txt« as file

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

execute_nvidia_smi: An error occurred: Command '['nvidia-smi', '--query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used', '--format=csv,noheader']' returned non-zero exit status 255.

Copy raw data to clipboard

Download »oo_errors.txt« as file

Copy raw data to clipboard

Download »progressbar« as file

2025-12-29 11:18:05 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, Started OmniOpt2 run...
2025-12-29 11:18:05 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, getting new HP set (no sbatch)
2025-12-29 11:18:05 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, requested 1 jobs, got 1, 0.07 s/job
2025-12-29 11:18:05 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, eval #1/1 start                    
2025-12-29 11:18:05 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, starting new job                   
2025-12-29 11:18:06 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, started new job                    
2025-12-29 11:18:16 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, new result: RESULT: -137118.664674 
2025-12-29 11:18:20 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, finishing jobs, finished 1 job
2025-12-29 11:18:20 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, getting new HP set (no sbatch)
2025-12-29 11:18:20 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, requested 1 jobs, got 1, 0.07 s/job
2025-12-29 11:18:21 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, eval #1/1 start                    
2025-12-29 11:18:21 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, starting new job                   
2025-12-29 11:18:21 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, started new job                    
2025-12-29 11:18:32 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, new result: RESULT: -51367.155862  
2025-12-29 11:18:34 (57454b0b-6448-45a1-bb13-28267c474e30): SOBOL, best RESULT: -137118.664674181, finishing jobs, finished 1 job     
2025-12-29 11:18:58 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, getting new HP set (no sbatch)
2025-12-29 11:18:58 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, requested 1 jobs, got 1, 24.02 s/job
2025-12-29 11:18:58 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, eval #1/1 start                     
2025-12-29 11:18:59 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, starting new job                    
2025-12-29 11:18:59 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, started new job                     
2025-12-29 11:19:09 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -137118.664674181, new result: RESULT: -285542.920000  
2025-12-29 11:19:11 (57454b0b-6448-45a1-bb13-28267c474e30): BOTORCH_MODULAR, best RESULT: -285542.92, finishing jobs, finished 1 job             

Copy raw data to clipboard

Download »progressbar« as file

Copy raw data to clipboard

Download »job_submit_durations.txt« as file

         Job submission durations          
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃   Batch ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│       1 │   0.197 │    1 │        0.197 │
│       2 │   0.194 │    1 │        0.194 │
│       3 │   0.180 │    1 │        0.180 │
├─────────┼─────────┼──────┼──────────────┤
│ Average │   0.190 │      │              │
│  Median │   0.194 │      │              │
│   Total │   0.571 │      │              │
│     Max │   0.197 │      │              │
│     Min │   0.180 │      │              │
└─────────┴─────────┴──────┴──────────────┘

Copy raw data to clipboard

Download »job_submit_durations.txt« as file

Copy raw data to clipboard

Download »generation_times.txt« as file

           Model generation times            
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┓
┃ Iteration ┃ Seconds ┃ Jobs ┃ Time per job ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━┩
│         1 │   0.070 │    1 │        0.070 │
│         2 │   0.072 │    1 │        0.072 │
│         3 │  24.017 │    1 │       24.017 │
├───────────┼─────────┼──────┼──────────────┤
│   Average │   8.053 │      │              │
│    Median │   0.072 │      │              │
│     Total │  24.159 │      │              │
│       Max │  24.017 │      │              │
│       Min │   0.070 │      │              │
└───────────┴─────────┴──────┴──────────────┘

Copy raw data to clipboard

Download »generation_times.txt« as file

Arguments Overview

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

timestamp,ram_usage_mb,cpu_usage_percent
1767003481,801.0859375,14.5

Copy raw data to clipboard

Download »cpu_ram_usage.csv« as file

RESULT (goal: minimize)

Best value: -285542.92
Achieved at:
- run_time = 7
- OO_Info_int_param = -76
- OO_Info_int_param_two = -100
- OO_Info_float_param = -100
- OO_Info_choice_param = 8
- int_param = -76
- float_param = -100
- int_param_two = -100
- choice_param = 8

Parameter statistics