LLM

FastAPI Resiliency: Circuit Breakers, Rate Limiting, and External API Management
Let’s assume that we are building a REST API services which depends on other 3rd party’s API services. To make things more realistic let’s assume that we are building a RAG using tavily (web search api) and LLMs.
Last updated on Mar 11, 2025 11 min read programming
FastAPI Resiliency: Circuit Breakers, Rate Limiting, and External API Management
On LLM reproducibility
One of the major issue with LLM output is its consistency and reproducibility. Here is kind of a hacky work around to make output from a LLM consistent (kind of). Also, a few observation regarding the same.
Last updated on Mar 11, 2025 7 min read programming
On LLM reproducibility