Inquiry About Intent Feature in OpenAI and Azure AI Search Integration

I am interested in understanding the logic behind the intent feature in the integration between OpenAI's chat completions and Azure AI Search. Specifically, I noticed that when appending and sending chat history for retrieval-augmented generation (RAG), there is a feature in the response named "intent." This output appears to be a reformulated query that is sent to the AI search to ensure it receives a meaningful request, allowing the search engine to complete its task effectively.

Take a look:

completion = client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Who is the first man to land on the moon?"},
        {"role": "assistant", "content": "The first man to land on the moon was Neil Armstrong."},
        {"role": "user", "content": "How old was he at that time?"}
    ],
    model=deployment,
    extra_body={
        "dataSources": [
            {
                "type": "AzureCognitiveSearch",
                "parameters": {
                    "endpoint": os.environ["SEARCH_ENDPOINT"],
                    "key": os.environ["SEARCH_KEY"],
                    "indexName": os.environ["SEARCH_INDEX_NAME"],
                }
            }
        ]
    }
)

Not Only the chat completion returns the correct answer which is 38 years old but also return an 'intent' feature :

print(completion.choices[0].message.context['intent'])

["How old was Neil Armstrong when he landed on the moon?", "What was Neil Armstrong's age when he landed on the moon?", "How old was Neil Armstrong when he was on the moon?"]

I’m very interested in understanding this mechanism, as I’m working with a stateless LangChain agent, and I would love to implement something similar.

I would like to know what prompt OpenAI uses to reformulate the query and send it to Azure AI Search using the "intent" feature. I want to be able to replicate this functionality using a prompt or some tools.

I am looking for a solution that allows me to summarize user inputs and send those reformulated queries to a LangChain agent. Is there any documentation or report that explains how to implement this intent feature in a standard chat completion? Additionally, are there any prompts used by OpenAI to reformulate and query the questions?

It is the detected intent from chat history, check this Getting intent considering all the chat history you are giving.

azure openai did not provided how it is creating intent publicly. So, you either use intent detection model or a prompt with examples to create intent.

Here is an example using prompt.

import json

messages=[
        {"role": "user", "content": "Who is the first man to land on the moon?"},
        {"role": "assistant", "content": "The first man to land on the moon was Neil Armstrong."},
        {"role": "user", "content": "How old was he at that time?"}
    ]

userquery = ','.join([json.dumps(i) for i in messages])

reformulation_prompt = """
You are an intelligent assistant tasked with helping users retrieve information from a database. The user may ask ambiguous, incomplete, or vague questions. Your task is to reformulate their query in a clear and precise way that can be sent to a search engine to retrieve the most relevant documents.

Here are some examples:
- User input: {"role": "user","content": "what is vector profiles?"}
  Reformulated queries: "What are vector profiles?", "Definition of vector profiles", "How do vector profiles work?"

Now, please reformulate the following user input, also give 2 to 3 Reformulated queries:
User input:""" + userquery + """
Reformulated query:
"""

t = client.completions.create(model=deployment,prompt=reformulation_prompt)
t.choices[0].text

and output:

Now you use this response in your langchain agent.