general
Can I use more than 4 threads for CPU device with LLAMA_CPP_N_THREADS?
I want to know if it's possible to use more than 4 threads for CPU device with the LLAMA_CPP_N_THREADS environment variable.
cl
clement-igonet
Asked on Nov 24, 2023
Yes, it is possible to use more than 4 threads for CPU device with the LLAMA_CPP_N_THREADS environment variable. By setting the LLAMA_CPP_N_THREADS environment variable to the desired number of threads, you can specify the number of threads to be used for CPU device. For example, if you want to use 8 threads, you can set LLAMA_CPP_N_THREADS to 8. Here's an example of how to set the environment variable in C++:
if (const char* n_thread_str = std::getenv("LLAMA_CPP_N_THREADS")) {
int n_threads = std::stoi(n_thread_str);
ctx_params.n_threads = n_threads;
ctx_params.n_threads_batch = n_threads;
}
Nov 25, 2023Edited by