It's time for LLMs to go "pro".

A year ago, it was okay to have fancy, shiny LLM prototypes, but now enterprises are expecting these technologies in production, at scale, and to be generating some real ROI.

One of the things that you have to manage as an LLM builder is the observability part. It's unreasonable to expect zero hallucinations. So you need a robust framework, platform, tools, processes, and governance to make sure you're monitoring these LLMs as much as possible so that if you do get some unexpected outcome, you at least know about it fast, and you can remedy the situation as quickly as possible.

I had an opportunity to talk to Joshua Rubin from Fiddler AI about the importance of LLM observability.