[data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxOSIgaGVpZ2h0PSIxOSIgYXJpYS1sYWJlbD0iVmlldyAxIFByaXZhdGUgTm90ZXMiPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgZD0iTTE0Ljc4IDguMDdhOC43IDguNyAwIDAgMC0uNDI3LTEuMzgzLjQ4LjQ4IDAgMCAwLS41ODQtLjI3bC0zLjEyLjc3VjQuMDM0YzAtLjI0Ny0uMTktLjQ4LS40My0uNWE3LjIgNy4yIDAgMCAwLTEuMzggMGMtLjI0LjAyLS40My4yNTMtLjQzLjVWNy4xOUw1LjMgNi40MTVhLjQ4LjQ4IDAgMCAwLS41ODMuMjdxLS4yNzEuNjczLS40MjYgMS4zODNjLS4wNS4yNC4xLjUuMzIuNThsMy4wNi43NTQtMS45OCAyLjk1NmMtLjE0LjE5Ni0uMTMuNTAyLjA0LjY3LjM0LjMzMi43LjYzMiAxLjA5Ljg5Ni4yLjEzNi40OS4wNzcuNjMtLjExN2wyLjA5LTMuMTE0IDIuMDkgMy4xMTJjLjE1LjE5My40My4yNTIuNjMuMTE2LjM5LS4yNi43NS0uNTYgMS4wOS0uODkuMTctLjE3LjE5LS40Ny4wNC0uNjdMMTEuNCA5LjQxbDMuMDYtLjc2YS41Mi41MiAwIDAgMCAuMzItLjU4IiAvPjwvc3ZnPg==](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxOSIgaGVpZ2h0PSIxOSIgYXJpYS1sYWJlbD0iVmlldyAxIFByaXZhdGUgTm90ZXMiPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgZD0iTTE0Ljc4IDguMDdhOC43IDguNyAwIDAgMC0uNDI3LTEuMzgzLjQ4LjQ4IDAgMCAwLS41ODQtLjI3bC0zLjEyLjc3VjQuMDM0YzAtLjI0Ny0uMTktLjQ4LS40My0uNWE3LjIgNy4yIDAgMCAwLTEuMzggMGMtLjI0LjAyLS40My4yNTMtLjQzLjVWNy4xOUw1LjMgNi40MTVhLjQ4LjQ4IDAgMCAwLS41ODMuMjdxLS4yNzEuNjczLS40MjYgMS4zODNjLS4wNS4yNC4xLjUuMzIuNThsMy4wNi43NTQtMS45OCAyLjk1NmMtLjE0LjE5Ni0uMTMuNTAyLjA0LjY3LjM0LjMzMi43LjYzMiAxLjA5Ljg5Ni4yLjEzNi40OS4wNzcuNjMtLjExN2wyLjA5LTMuMTE0IDIuMDkgMy4xMTJjLjE1LjE5My40My4yNTIuNjMuMTE2LjM5LS4yNi43NS0uNTYgMS4wOS0uODkuMTctLjE3LjE5LS40Ny4wNC0uNjdMMTEuNCA5LjQxbDMuMDYtLjc2YS41Mi41MiAwIDAgMCAuMzItLjU4IiAvPjwvc3ZnPg==)

If you’re working in Generative AI, chances are you’re using a language model in your day-to-day work. It could be Azure OpenAI, Anthropic’s Cluade or even one from AWS Bedrock. Regardless of which LLM it is there is a great deal one can learn from analyzing the actions we take and their responses.

This guide will be focusing on debugging Azure OpenAI calls made through Langchain

Before we being be sure to install the following packages

pip install -U -q langchain langchain_openai langfuse

Let us start by first defining our LLM

from langchain_openai import (AzureChatOpenAI
import os

llm = AzureChatOpenAI(
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    deployment_name=os.getenv("DEPLOYMENT_NAME"),
    api_key=getenv("AZURE_OPENAI_API_KEY"),
    api_version=os.getenv("OPENAI_API_VERSION"),
    temperature=0.4, max_tokens=1000,
)

We will be using Langfuse an open-source LLM tracing application to monitor our LLM usage. LangSmith would have been my first choice but they no longer provide a self-hosted option.

Langfuse

Let’s first configure Langfuse locally, you will need Docker for this.

git clone <https://github.com/langfuse/langfuse.git>
cd langfuse
docker compose up

Now you can access your local langfuse at http://localhost:3000/