How to resolve the 'invalid magic characters' error when running a model with cargo on WSL Ubuntu?
I'm trying to build and run a project on WSL Ubuntu by following the CONTRIBUTING.md from the main branch. However, when I execute the command cargo run --features cuda serve --model TabbyML/StarCoder-1B --device cuda, I encounter an error message that says gguf_init_from_file: invalid magic characters 'Inva'. I'm looking for some insight into this issue.
ZIHAN WANG
Asked on Mar 07, 2024
It seems like the issue might be related to a known problem with the huggingface registry mirror. A potential workaround is to set the TABBY_DOWNLOAD_HOST environment variable to a different mirror and then run the download command again. For example, you can try using TABBY_DOWNLOAD_HOST=http://modelscope.cn|modelscope.cn cargo run download --model TabbyML/StarCoder-1B. If you're using the cargo run command instead of a pre-built tabby binary, make sure to include the environment variable in the command line. Also, ensure that you're not using the TABBY_DOWNLOAD_HOST variable for models that are not related to the starcoder, as it might cause other issues.