How can I switch to GPU in Tabby?

I'm a new user of Tabby and I'm facing a problem with slow performance. It suggests switching to GPU, but I'm not sure how to do that. Can someone help me?


Gael Denysiak

Asked on Dec 08, 2023

To switch to GPU in Tabby, you need a NVIDIA GPU and install the NVIDIA Container Toolkit with Docker. Once you have the toolkit installed, you can use the sample command provided in the Tabby documentation to run your model with GPU acceleration.

Dec 08, 2023Edited by