nlpmistral-7b

Are [INST] and [/INST] needed for mistral chat?


On the page Open-weight models, they use [INST] and [/INST] in the template. (If I understood correctly, they fined tuned the basic text completion model on a dataset of instructions with the format <s>[INST] Instruction [/INST] Model answer</s>[INST] Follow-up instruction [/INST] so that you can "turn on/off" the instruction mode.)

However, in the guide for RAG, they do not use these special tokens, so I am confused about whether they are necessary. Is the instruct model being used or is there another model fined tuned on chat messages?

I found this page Fine-tuning Mistral 7B for chat, which suggests that the instruct model can be used for chat.

To show the generalization capabilities of Mistral 7B, we fine-tuned it on instruction datasets publicly available on HuggingFace. No tricks, no proprietary data. The resulting model, Mistral 7B Instruct, outperforms all 7B models on MT-Bench, and is comparable to 13B chat models.


Solution

  • Mistral got two different models, Mistral-7B and Mistral-7B-Instruct.The Mistral-7B-Instruct is fine-tuned for conversation and question answering. For Instruct you need to put [INST] and same as you already mentioned. But for Mistral-7B you can give any format and it will try to generate. Also, if you are using transformers library you can use apply_chat_template (https://huggingface.co/docs/transformers/main/en/chat_templating).

    for more details about mistral, you can visit https://www.promptingguide.ai/models/mistral-7b