I am tweaking a python script using diffusers inpainting pipeline for a custom video generation idea.
I would like to gradually shift the weights of certain words in the prompt.
As I understand the argument prompt_embeds is exactly what i need.
I could not figure out how to define this argument. Can someone pls provide an example?
I know there are frameworks out there where you can just add weights to certain words with the following syntax:
"This is a SD prompt with plus 50% weight added to the last (word:1.5)"
This would be a great solution as well however this does not work with diffusers.StableDiffusionInpaintPipeline
Example from the original repo of Compel library:
from diffusers import StableDiffusionPipeline
from compel import Compel
pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)
# upweight "ball"
prompt = "a cat playing with a ball++ in the forest"
conditioning = compel.build_conditioning_tensor(prompt)
# or: conditioning = compel([prompt])
# generate image
images = pipeline(prompt_embeds=conditioning, num_inference_steps=20).images
images[0].save("image.jpg")
So, as you can see, you should add ++, or --, or ----- and so on.
For your case it would be like "This is a SD prompt with plus 50% weight added to the last word+++++". Also you can up\downweight subprompt: A rich and descriptive (a subprompt goes here)++++ prompt.