pythonlangchainlarge-language-modelollama

How to view the final prompt in a MultiQueryRetriever pipeline using LangChain?


I am currently working on a project using the LangChain library where I want to retrieve relevant documents from a vector database and then generate answers based on these documents using the Ollama LLM.

Below is my current implementation:

import logging

logging.basicConfig()
logging.getLogger("langchain.retrievers.multi_query").setLevel(logging.INFO)

# Define the prompt template for generating multiple query versions
QUERY_PROMPT = PromptTemplate(
    input_variables=["question"],
    template="""You are an AI language model assistant. Your task is to generate five
    different versions of the given user question to retrieve relevant documents from
    a vector database. By generating multiple perspectives on the user question, your
    goal is to help the user overcome some of the limitations of the distance-based
    similarity search. Provide these alternative questions separated by newlines.
    Original question: {question}""",
)

# Initialize the MultiQueryRetriever
retriever = MultiQueryRetriever.from_llm(
    vectordb.as_retriever(), 
    ollama,
    prompt=QUERY_PROMPT
)

# Modified RAG prompt for generating the final response
template = """Answer the question based ONLY on the following context:
{context}
Question: {question}
"""

# Create the final QA chain
prompt = ChatPromptTemplate.from_template(template)

from langchain_core.runnables import RunnableLambda


def inspect(state):
    """Print the state passed between Runnables in a langchain and pass it on"""
    print(state)
    return state


qa_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | RunnableLambda(inspect)  # Add the inspector here to print the intermediate results
    | prompt
    | ollama
    | StrOutputParser()
)

# Invoke the QA chain with a sample query
qa_chain.invoke("Give 10 quotes from this articles related to love?")

How can I view the final prompt that is generated by the qa_chain before it is sent to the Ollama LLM for processing? I would like to see the exact prompt that includes the context and the user's question.


Solution

  • Enable verbose and debug mode. For example,

    ...
    from langchain_core.runnables import RunnableLambda, RunnablePassthrough
    from langchain.schema.output_parser import StrOutputParser
    
    from langchain.globals import set_verbose, set_debug
    
    set_debug(True)
    set_verbose(True)