general

How to resolve GGML_ASSERT error when running CodeLlama on MacBook M1?

I'm trying to run CodeLlama on my MacBook M1 using the tabby serve command, but I'm getting a GGML_ASSERT error. I have 16GB of RAM. Any idea how to resolve this?

cl

clement-igonet

Asked on Nov 19, 2023

The GGML_ASSERT error is likely caused by running out of memory (OOM). Since you have 16GB of RAM, it's possible that the model you're trying to load requires more memory than what is available. You can try the following steps to resolve the issue:

  1. Reboot your MacBook to free up memory.
  2. Close any unnecessary applications or processes running in the background to free up additional memory.
  3. If the issue persists, you may need to upgrade your RAM to a higher capacity.

Keep in mind that running large models like CodeLlama may require a significant amount of memory, so it's important to ensure that your system meets the minimum requirements.

Nov 21, 2023Edited by