What is "Query" and How Is It Calculated?

Modified on Fri, 19 Jan, 2024 at 2:04 AM

On Vext, "Query" is a basic usage unit that calculates how many times an LLM has been triggered. We decided not to use "token" because it could be confusing for some and difficult to manage/forecast usage and cost.


For example, if you have a project with one LLM configured and is integrated with your app via API, when there's a POST request to the API, it's considered 1 query.


Where there are multiple LLMs configured in a project, say three LLMs within the same project, and when invoked, it's considered 3 queries.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article