Snowshoe logo
general

How to optimize CPU usage when launching Tabby server?

I'm wondering if it's possible to apply a setup to optimize CPU usage when launching the Tabby server. Clement mentioned about exploiting M3 macOS CPU with LM Studio, and shared the command used to start Tabby server along with the output of /v1/health. Meng Zhang suggested trying to use 1.3B deepseekcoder for --model.

cl

clement-igonet

Asked on Feb 27, 2024

  • To optimize CPU usage when launching the Tabby server, you can try using a specific model for the --model parameter.
  • In this case, using the 1.3B deepseekcoder model may help improve CPU performance.
  • Additionally, ensure that the GPU layers are already maximized for efficient processing.

Example command with optimized --model parameter:

$ TABBY_DISABLE_USAGE_COLLECTION=1 tabby serve --device metal --model TabbyML/DeepseekCoder-1.3B --chat-model TabbyML/Mistral-7B --webserver
Feb 28, 2024Edited by