Tabby Community - How can I optimize the usage of Metal when running Tabby on Mac M1?

general

How can I optimize the usage of Metal when running Tabby on Mac M1?

I'm using the --device metal option to run Tabby on my Mac M1, but the GPU usage does not go above 8% and it takes 20-30 seconds to get results. Is there any optimization I could apply to improve this?

cl

clement-igonet

Asked on Feb 26, 2024

Make sure you have the latest version of Tabby installed.
Check if the precompiled binary you are using is optimized for Metal.
Verify that Metal is being utilized properly when running Tabby.
Consider optimizing your code for better GPU utilization.
Experiment with different configurations and parameters to find the optimal settings for your setup.

Feb 26, 2024Edited by