general
How can I build and run tabby with CUDA support?
Sa
Sam Tenenholz
Asked on Dec 18, 2023
To build and run tabby with CUDA support, you can use the following command: ./target/debug/tabby serve --device cuda --model TabbyML/StarCoder-1B
. This will run the tabby binary with the specified device set to CUDA and the model set to TabbyML/StarCoder-1B. Alternatively, you can use cargo run -F cuda serve --device cuda --model TabbyML/StarCoder-1B
to achieve the same result.
Dec 19, 2023Edited by