py311-bitsandbytes

k-bit optimizers and matrix multiplication routines

bitsandbytes enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training.