general

Why does a trigger lead to multiple generate requests to LLM models in manual mode with ollama as the backend?

In manual mode, when using ollama as the backend, why does a trigger lead to multiple generate requests to LLM models? The user noticed that 3 requests are sent and only the first one is used as the completion.

Er

Ervin Xie

Asked on Jul 03, 2024

  • In manual mode with ollama as the backend, a trigger leads to multiple generate requests to LLM models to provide multiple choices for completion.

  • To choose other choices, you can hover over a suggestion to show the toolbar, or press Alt + [ or Alt + ] to cycle through the suggestions.

  • The manual trigger always requests multiple choices, while in auto-trigger mode, a single choice is requested automatically, and multiple choices are requested when hovering or cycling is triggered, which may cause a slight delay.

Aug 29, 2024Edited by