google-cloud-platformgoogle-apipalm-apimakersuite

How to Use images as input for palm api?


I am currently exploring the Palm2 API for image processing, but I am facing challenges due to the lack of comprehensive documentation in the makersuite documentation. I have reviewed the available resources, but couldn't find specific information on how to use Palm2 for image-related tasks.

I would appreciate it if someone with experience in using Palm2 could provide guidance or point me to any unofficial documentation, tutorials, or examples that might be available. Specifically, I am interested in performing specific image processing tasks you want to achieve, e.g., image recognition, feature extraction, etc.


Solution

  • Palm api doesnt offer image processing at this time. It just has

    if you want to do image processing I think you should have a look at vertex AI

    PROJECT_ID = "PROJECT_ID"  # @param {type:"string"}
    LOCATION = "LOCATION"  # @param {type:"string"}
    
    import vertexai
    from vertexai.vision_models import ImageTextModel, Image
    
    vertexai.init(project=PROJECT_ID, location=LOCATION)
    model = ImageTextModel.from_pretrained("imagetext@001")
    
    source_image = Image.load_from_file(location='./gen-img1.png')
    
    answers = model.ask_question(
        image=source_image,
        question="What breed of dog is this a picture of?",
        # Optional:
        number_of_results=2,
    )
    print(answers)