How can I make sure that Tabby is using AMD GPU?

I'm running Tabby with an AMD GPU (6700XT) and it seems to be running slow. How can I verify that Tabby is using the GPU?



Asked on Jan 08, 2024

You can check if Tabby is using the GPU by looking at the console output when running the Tabby server. It should mention whether it detected a GPU. In your case, the console output you shared does not mention anything about a GPU. This could indicate that Tabby is not utilizing the GPU. You can try the following steps to troubleshoot:

  1. Make sure you have installed the correct GPU drivers for your AMD GPU.

  2. Verify that you have downloaded the RoCm build of Tabby from the correct source.

  3. Double-check the command you are using to start the Tabby server and ensure that the --device rocm flag is correct.

  4. Check if there are any error messages or warnings related to the GPU in the console output.

If you have followed these steps and Tabby still does not seem to be using the GPU, you may need to reach out to the Tabby team for further assistance.

Jan 13, 2024Edited by