Flux.1 is the newest text-to-image model in the market, brought to us by Black Forest Labs. It is a state-of-the-art model that can generate high-quality images from text descriptions handling complex descriptions and generating high-quality images with fine details.
Who is behind Flux.1?
Flux.1 is developed by Black Forest Labs, a company created by a group of ex employees from Stability AI.
How does it work?
Unlike other diffusion models, like Stable Diffusion that create images by gradually removing noise from a random start point, Flux.1 generates images using a technique called "flow matching" that takes a more direct approach, learning the exact transformations needed to convert noise into a realistic image. This allows to generate high-quality images faster and with less steps than common diffusion models.
Also, with this different approach, Flux.1 can handle images with text inside it, like the one below:
A photorealistic image of a modern, sleek laptop with a webpage open displaying the text "codestackme" in a clean, minimalist design. The laptop should be positioned on a white desk with soft lighting, highlighting the screen's glow and the subtle reflections on the metallic casing. The overall atmosphere should be professional and inviting, conveying a sense of innovation and technological advancement.
How to write a good prompt for Flux.1?
One of Flux.1’s standout features is its user-friendly prompting mechanism. The integration of CLIP (from OpenAI) and T5 (from GoogleAI) text encoders allows the model to interpret descriptions with a high degree of nuance. CLIP excels in aligning text with visual content, while T5 enhances the model's ability to process structured text inputs. Together, they enable Flux.1 to generate images that closely match the detailed prompts provided by users.
What types of models are there for Flux.1?
Flux.1 comes in three distinct versions: Schnell, Dev and Pro.
- Schnell is the fastest model, optimized for speed and efficiency. It is allowed for commercial use, since was released under the Apache 2.0 license.
- Dev provides a more flexible and experimental framework, it’s focused for developers and researches who want to fine-tune or customize certain features of the model. It was released with a non-commercial license.
- Pro is the most advanced and resource-intensive version. It offers higher resolution outputs and can generate more complex images, however it is only available though the Black Forest Labs API.
How to use Flux.1 for free?
For those interested in exploring the capabilities of Flux.1 without financial commitment, using modal.com as a resource provider is a viable option. Modal.com offers a monthly compute power allowance of $30, which can support the generation of numerous images each month. You can learn more about their pricing and offerings at Modal.com Pricing.
This recommendation is not sponsored or endorsed by the platform.
To begin, you'll first need to create an account on modal.com by logging in using your GitHub credentials.
Next, you'll need to install the Modal CLI. Ensure that Python is installed on your computer. Once Python is set up, open your terminal and execute the command pip install modal. After the installation is complete, run modal setup to link the CLI with your Modal account.
Proceed by cloning this GitHub repository to your computer and navigate to the cloned directory.
For security, create a secret called flux.1-secret in your modal dashboard with an environment variable named API_KEY and assign it a random string.
Finally, deploy your service by running modal deploy app.py --name flux1 in your terminal. Upon successful deployment, modal will provide a URL for accessing the web service:
✓ Created objects.
├── 🔨 Created mount PythonPackage:app
├── 🔨 Created function Model.build.
├── 🔨 Created function Model.*.
├── 🔨 Created function Model._inference.
└── 🔨 Created web function Model.web_inference => <PUBLIC_URL>
✓ App deployed in 3.206s! 🎉
To use the service, make a GET request to the provided PUBLIC URL. Include the x-api-key you set earlier in the headers, and encode your prompt in the query parameters. You can also specify desired image dimensions through query parameters. Here’s an example of how to structure your request:
curl -H "x-api-key: <API_KEY>" <PUBLIC_URL>?width=<WIDTH>&height=<HEIGHT>&prompt=<PROMPT>
Understanding the Code
Let's dissect the app.py file, which is crucial for running our Flux.1 image generation service using modal's platform. Here's a breakdown of the setup and functionality:
import modal
image = modal.Image.debian_slim(python_version="3.10").apt_install(
"libglib2.0-0",
"libsm6",
"libxrender1",
"libxext6",
"ffmpeg",
"libgl1",
"git"
).pip_install(
"git+https://github.com/huggingface/diffusers.git",
"invisible_watermark",
"transformers",
"accelerate",
"safetensors",
"sentencepiece",
)
This block defines the Docker image for our application, specifying the OS, necessary libraries, and Python packages. This environment supports the execution of the Flux.1 model and associated utilities.
app = modal.App('flux1')
with image.imports():
import os
import io
import torch
from diffusers import FluxPipeline
from fastapi import Response, Header
Here, we initialize our app and import necessary Python libraries within the context of our previously defined Docker image. These imports are essential for image processing and handling web requests.
@app.cls(gpu=modal.gpu.A100(), container_idle_timeout=15, image=image, timeout=120, secrets=[modal.Secret.from_name("flux.1-secret")])
class Model:
@modal.build()
def build(self):
from huggingface_hub import snapshot_download
snapshot_download("black-forest-labs/FLUX.1-schnell")
@modal.enter()
def enter(self):
print("Loading model...")
self.pipeline = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to('cuda')
print("Model loaded!")
def inference(self, prompt: str, width: int = 1440, height: int = 1440):
print("Generating image...")
image = self.pipeline(
prompt,
output_type='pil',
width=width,
height=height,
num_inference_steps=8,
generator=torch.Generator("cpu").manual_seed(
torch.randint(0, 1000000, (1,)).item()
)
).images[0]
print("Image generated!")
byte_stream = io.BytesIO()
image.save(byte_stream, format="PNG")
return byte_stream.getvalue()
@modal.web_endpoint(docs=True)
def web_inference(self, prompt: str, width: int = 1440, height: int = 1440, x_api_key: str = Header(None)):
api_key = os.getenv("API_KEY")
if x_api_key != api_key:
return Response(content="Unauthorized", status_code=401)
image = self.inference(prompt, width, height)
return Response(content=image, media_type="image/png")
This section defines the main functionality of our service:
-
@modal.build(): Downloads the model when the application builds. -
@modal.enter(): Loads the model into GPU memory the first time the service is invoked. -
@modal.web_endpoint(): Serves as the web endpoint for our service using FastAPI.
If you just want to run it as a local service, you can add @modal.method() and define it as following inside the class.
@modal.method()
def _inference(self, prompt: str, width: int = 1440, height: int = 1440):
return self.inference(prompt, width, height)
And outside it, define a local entry point
@app.local_entrypoint()
def main(prompt: str = "A beautiful sunset over the mountains"):
image_bytes = Model()._inference.remote(prompt)
with open("output.png", "wb") as f:
f.write(image_bytes)
Local entry point will run locally on your machine calling the _inference method remotely, so you still using the modal’s service, without exposing it to the internet.
Conclusion
Flux.1 is not just another tech breakthrough - it's a game-changer for anyone who's ever dreamed of bringing their ideas to life visually. Imagine being able to describe a scene in words and watch as it materializes into a stunning, detailed image right before your eyes. That's the magic of Flux.1. It's like having a super-talented artist at your fingertips, ready to paint your thoughts with incredible precision. Whether you're an artist looking to speed up your creative process, a designer in need of quick visual concepts, or just someone who loves playing with new tech, Flux.1 opens up a world of possibilities. It's not about replacing human creativity - it's about enhancing it, making the journey from imagination to reality smoother and more exciting than ever before.
Top comments (1)
Thanks for sharing the results of your studying and your knowledge in such a well written and concise text.
I hope getting more from you in the future! :)