How does a chat application with multiple users handle memory and context?
For example:
User 1 asks questions about Java programming. The application should retain and use User 1's specific conversation history and context for accurate responses.
User 2 asks questions about Python programming. The application should retain and use User 2's specific conversation history and context for accurate responses.
How does Spring AI ensure that the correct memory or conversation history is associated with the corresponding user's prompt?
For instance, when User 1 asks a Java-related query, the system should reference only User 1's conversation history and not confuse it with User 2's context.
There is ChatMemory interface for it and it has implementation provided by spring boot like InMemoryChatMemory, CassandraChatMemory. if you need own implementation than you can do by implementing ChatMemory interface. and pass this implementation to ChatClient as advisor like MessageChatMemoryAdvisor. this interface uses unique conversation id to fetch user's memory and add it to user prompt this is useful for storing small term conversation because there is token limit for LLM what you can send.
Example:
chatClient = builder
.defaultAdvisors(
new MessageChatMemoryAdvisor(chatMemory));
Now when you call method you can pass conversationId unique to every chat session like this:
chatClient
.prompt()
.advisors(advisorSpec -> advisorSpec.param(CHAT_MEMORY_CONVERSATION_ID_KEY, {conversationId}))
.user({userMessage})
.call().content();
There is also VectorStoreChatMemoryAdvisor that can retrieve information from vector store which is useful for long term conversations. It will find information from vectorstore with similarity search and only add necessary info to prompt.
Here is link to Spring AI project I created you can refer to it for all this: https://github.com/KushParsaniya/notebook-llm.