I have some words that I do not want produced by Claude. No matter how I prompt it, there is a non zero chance that it will still get produced in the output. So I want to make it stop when it produces such a word instead of wastefully producing long sequences which I will filter out anyway and wasting money. Is there a way to do this?
I have a basic prompt like this so far
payload = {
"prompt": "Tell me a 500 word joke but make sure it doesn't include the word 'blood' ",
"model": "claude-3-5-sonnet-20240620",
"max_tokens_to_sample": 50
}
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
response = requests.post(API_ENDPOINT, json=payload, headers=headers)
I tried running it with the prompt but I notice that sometimes the word 'blood' still gets produced in 500 word jokes which I have to trim or ignore so it's a waste of resources.
See “stop_sequences” in the API. But I don’t know for sure if it will reduce your resource costs by a lot. Better to use strong prompting and maybe even fine tuning.
https://docs.anthropic.com/en/api/complete
stop_sequences string[] Sequences that will cause the model to stop generating.
Our models stop on "\n\nHuman:", and may include additional built-in stop sequences in the future. By providing the stop_sequences parameter, you may include additional strings that will cause the model to stop generating.