AI-Driven Art Analysis and Generation with Lyzr Automata

In today's digital age, the fusion of artificial intelligence (AI) and art has opened up new avenues for creativity and expression. Leveraging the power of Lyzr Automata, an innovative platform for building AI-driven solutions, we embark on a journey to develop an Art Analyzer and Generator that pushes the boundaries of artistic exploration.
Art has always been a medium for human expression, and with advancements in AI technology, we can now delve deeper into the realm of creativity. In this blog post, we'll walk through the process of creating an Art Analyzer and Generator using Lyzr Automata, Streamlit for the user interface, and OpenAI's state-of-the-art models.

Here's a walk-through for the Lyzr Art Analyzer:
Imports and Setup:

import base64
import shutil
import streamlit as st
from openai import OpenAI
from lyzr_automata.ai_models.openai import OpenAIModel
from lyzr_automata import Agent, Task
from lyzr_automata.tasks.task_literals import InputType, OutputType
from lyzr_automata.pipelines.linear_sync_pipeline  import  LinearSyncPipeline
from dotenv import load_dotenv,find_dotenv
from PIL import Image
import os

base64: Used for encoding images to base64 format.
shutil: Used for file management (deleting existing files).
streamlit: Used for building the Streamlit web app interface.
openai: Used for interacting with the OpenAI API (deprecated, replaced with client from lyzr_automata).
lyzr_automata: Used for building and running the Lyzr agent pipeline, including OpenAIModel, Agent, Task, etc.
dotenv: Used for loading environment variables from a .env file.
os: Used for interacting with the operating system.

load_dotenv(find_dotenv())
api = os.getenv("OPENAI_API_KEY")

client = OpenAI()

load_dotenv is called to load environment variables from a .env file, likely containing the OpenAI API key.

Image Upload and Processing:

def remove_existing_files(directory):
    for filename in os.listdir(directory):
        file_path = os.path.join(directory, filename)
        try:
            if os.path.isfile(file_path) or os.path.islink(file_path):
                os.unlink(file_path)
            elif os.path.isdir(file_path):
                shutil.rmtree(file_path)
        except Exception as e:
            print(e)

data_directory = "data"
os.makedirs(data_directory, exist_ok=True)
remove_existing_files(data_directory)

uploaded_file = st.sidebar.file_uploader("Choose PDF file", type=["jpg",'png','webp'])

A function remove_existing_files is defined to remove any existing files in the data directory before uploading new ones.
A data directory named "data" is created using os.makedirs.

st.sidebar.file_uploader is used to create a file upload section in the sidebar, allowing users to select a JPG, PNG, or WEBP image.

if uploaded_file is not None:
    # Save the uploaded PDF file to the data directory
    file_path = os.path.join(data_directory, uploaded_file.name)
    with open(file_path, "wb") as file:
        file.write(uploaded_file.getvalue())

    # Display the path of the stored file
    st.sidebar.success(f"File successfully saved")

If a file is uploaded:
The file is saved to the data directory using its original name.
A success message is displayed using st.sidebar.success.

def get_all_files(data_directory):
    # List to store all file paths
    file_paths = []

    # Walk through the directory tree
    for root, dirs, files in os.walk(data_directory):
        for file in files:
            # Join the root path with the file name to get the absolute path
            file_path = os.path.join(root, file)
            # Append the file path to the list
            file_paths.append(file_path)
            break

    return file_paths

A function get_all_files is defined to retrieve all file paths within the data directory.

def image_to_base64(image_path):
    with open(image_path, "rb") as image_file:
        image_bytes = image_file.read()
        base64_encoded = base64.b64encode(image_bytes)
        return base64_encoded.decode('utf-8')

A function image_to_base64 is defined to convert an image at a given path to a base64 encoded string.

Image Analysis:

def generate_img_insights(image):
    response = client.chat.completions.create(
      model="gpt-4-vision-preview",
      messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": """Describe image style and give insights of an image.give in below format:
            Art Style: 
            Insights:
            """},
            {
              "type": "image_url",
              "image_url": {
                "url": f"data:image/jpeg;base64,{base64_image}",
              },
            },
          ],
        }
      ],
      max_tokens=300,
    )

    return response.choices[0].message.content

generate_img_insights function takes an image (base64 encoded):
Uses the client.chat.completions.create method to send an instruction and the image to the OpenAI API.
The instruction specifies describing the image style and providing insights in a specific format.
Returns the generated response containing insights about the image.

Image generation:

def generate_image(insights):
    artist_agent = Agent(
            prompt_persona="You are an Artist and you generate image based on art style and insights",
            role="Artist",
        )

    open_ai_model_image = OpenAIModel(
        api_key=api,
        parameters={
            "n": 1,
            "model": "dall-e-3",
        },
    )

    art_generation_task = Task(
            name="Art Image Creation",
            output_type=OutputType.IMAGE,
            input_type=InputType.TEXT,
            model=open_ai_model_image,
            log_output=True,
            instructions=f"""Generate an image with below instructions:
            {insights}
            """,
        )

    output = LinearSyncPipeline(
            name="Generate Art",
            completion_message="Art Generated!",
            tasks=[
                art_generation_task
            ],
        ).run()
    return output[0]['task_output'].url

generate_image function takes the insights string:
Creates a Lyzr Agent named "Artist" with a persona describing its role.
Creates an OpenAI model object for image generation using dall-e-3.
Defines a Task named "Art Image Creation" that specifies:
Output type as image.
Input type as text (the insights).
The model to use (OpenAI model for image generation).
Instructions for generating an image based on the provided insights.
Creates a LinearSyncPipeline named "Generate Art" that runs the defined task and outputs the generated image URL.
Returns the URL of the generated image.

Processing and Output:

path = get_all_files(data_directory)
if path is not None:
    for i in path:
        st.sidebar.image(i)
        base64_image = image_to_base64(i)
        insights = generate_img_insights(base64_image)
        images = generate_image(insights)
        if images:
            st.markdown(insights)
            st.image(images)

get_all_files is called to get a list of all image paths in the data directory.
If there are files:
A loop iterates through each image path:
The image is displayed in the sidebar using st.sidebar.image.
The image is converted to base64 format using image_to_base64.
generate_img_insights is called to get insights about the image.
generate_image is called with the insights to generate a new image based on those insights.
If an image URL is returned:
The insights are displayed using markdown.
The generated image is displayed using st.image with the URL.

Ready to explore the world of AI-driven art analysis and generation? Visit our deployed Streamlit application or run the provided code to experience the Lyzr Art Analyzer firsthand. Upload your favorite artworks, generate insights, and witness the AI's creative output in action. Join us on this journey of artistic exploration, fueled by the power of Lyzr Automata and AI innovation.

try it now:https://lyzr-art-explorer.streamlit.app/

In summary, the collaboration between AI and art opens up a world of endless possibilities. With Lyzr Automata, we empower creators to push the boundaries of creativity, fostering a new era of artistic expression and exploration. Join us as we embark on this exciting journey at the forefront of AI-driven creativity.

For more information explore the website: Lyzr