GenAI

Let’s assume that we are building a REST API service that depends on third-party APIs. To make the example more realistic, let’s say we are building a RAG system that uses Tavily (a web search API) and LLMs.

One of the major issue with LLM output is its consistency and reproducibility. Here is kind of a hacky work around to make output from a LLM consistent (kind of). Also, a few observation regarding the same.