general
Does Tabby support GPU offloading?
I want to achieve decent performance/low delay while using DeepSeekCoder 6.7B with 8GB VRAM. The model mentions that using GPU offloading will lower RAM consumption. However, I'm using an AMD GPU. Does Tabby support GPU offloading?
Ri
RiQuY
Asked on Apr 11, 2024
- GPU offloading in Tabby is currently only possible for CUDA-based backends.
- Tabby does not support GPU offloading for AMD GPUs at the moment.
Apr 11, 2024Edited by