Image Analysis with Amazon Bedrock and Anthropic Claude 3

This blog delves into image analysis techniques using Amazon Bedrock and Anthropic Claude 3.

This is simple example of emonstrating the use of Amazon Bedrock and Multi-Modal Generative AI models from Anthropic to implement an image analysis use case.

The application is constructed with a simple streamlit frontend where users can upload a 1 page jpeg, png or PDF and get a description of the image.

Follow the below simple steps in order to use Amazon bedrock.

Step 1: Clone the Github repo for GenAI quickstart PoCs

https://github.com/aws-samples/genai-quickstart-pocs

Go to amazon-bedrock-claude3-image-analysis-poc directory.

The file structure of this repo is broken into 3 key files

1. app.py file- Frontend application (a streamlit app)
2. analyze_images.py — Amazon Bedrock API invocations to generate descriptions of an image
3. requirements.txt — contains all necessary dependencies for this sample application to work.

Step 2: Set up a python virtual environment

pip install virtualenv
python3.10 -m venv venv

Step 3: Activate virtual environment

cd venv
cd bin
source activate
cd ../../

Step 4: install all the requirements

pip install -r requirements.txt

Step 5: Export AWS profile

profile_name=<AWS_CLI_PROFILE_NAME>

Step 6: Run the app

streamlit run app.py

You should see the below screen where you can upload the image.

Let’s see the code used

System Prompt

Describe every detail you can about this image, be extremely thorough 
and detail even the most minute aspects of the image.

Primary prompt passed to Claude3, user uploaded image in base64 and any other text

prompt = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "temperature": 0.5,
        "system": system_prompt,
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": file_type,
                            "data": image_base64
                        }
                    },
                    {
                        "type": "text",
                        "text": text
                    }
                ]
            }
        ]
    }

Invoke Bedrock

response = bedrock.invoke_model(body=json_prompt, 
                    modelId="anthropic.claude-3-sonnet-20240229-v1:0",
                    accept="application/json", contentType="application/json")

This was a basic setup, but you can easily modify the code to change the model and other parameters.

If you have any questions, please feel free to reach out to me on X