Change default CHGnet.load(check_cuda_mem: bool) to False

[janosh]

there's a problem with cuda_devices_sorted_by_free_mem on slurm clusters

chgnet/chgnet/utils/common_utils.py

Lines 36 to 50 in 81439f2

	def cuda_devices_sorted_by_free_mem() -> list[int]:
	"""List available CUDA devices sorted by increasing available memory.
	To get the device with the most free memory, use the last list item.
	"""
	if not torch.cuda.is_available():
	return []
	free_memories = []
	nvidia_smi.nvmlInit()
	device_count = nvidia_smi.nvmlDeviceGetCount()
	for idx in range(device_count):
	handle = nvidia_smi.nvmlDeviceGetHandleByIndex(idx)
	info = nvidia_smi.nvmlDeviceGetMemoryInfo(handle)
	free_memories.append(info.free)

it will return whatever GPU has most free memory and so the model tries to use that even if the job was allocated a different GPU. this results in a cryptic CUDA error

    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
RuntimeError: CUDA error: invalid device ordinal

Process finished with exit code 1

given CHGNet is expected to often be used on queued HPC infra where this error can happen and the error message is not obvious to debug, @BowenD-UCB and I agreed to change the default from True to False