Hi,
I see that bitsandbytes binaries are available for cuda 11.8 and 12.0 only. How can I get a compatible version for cuda 12.1, because I see this message which it can not find the 12.1 library file. I have to say that the same library name for 12.0 exists.
$ python -m bitsandbytes
Could not find the bitsandbytes CUDA binary at PosixPath('/home/mahmood/.conda/envs/llama3-train/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so')
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.