How can I build and run tabby with CUDA support?


Sam Tenenholz

Asked on Dec 18, 2023

To build and run tabby with CUDA support, you can use the following command: ./target/debug/tabby serve --device cuda --model TabbyML/StarCoder-1B. This will run the tabby binary with the specified device set to CUDA and the model set to TabbyML/StarCoder-1B. Alternatively, you can use cargo run -F cuda serve --device cuda --model TabbyML/StarCoder-1B to achieve the same result.

Dec 19, 2023Edited by