general

Why is the Tabby server hanging when running without Docker?

I ran the Tabby server locally without Docker using cargo build, but it's just hanging without showing 'Listening at 0.0.0.0:8080'. What could be the reason?

Ry

Ryan Y

Asked on Dec 18, 2023

The Tabby server may hang when running without Docker due to a CUDA version issue. It seems that the system is missing the CUDA 11 related libraries, which are required by the Tabby server. To resolve this issue, you can either use the Docker image or try using the manylinux CUDA binary provided in the TabbyML GitHub repository. Additionally, a future release of Tabby may include a manylinux2014 CUDA 12 binary to address this issue.

Dec 18, 2023Edited by