DEV Community

Cover image for I built an AI room designer: the hard part was turning taste into controls
Jammy Lee
Jammy Lee

Posted on AI-assisted

I built an AI room designer: the hard part was turning taste into controls

Ask someone how they want to redesign their living room and you'll rarely hear something that could go straight into a good AI prompt.

Instead, you'll hear things like:

"I want it warmer, but not too minimal."

Or:

"Something like Japandi, but more comfortable."

Or even:

"I don't know what style I want. I just want this room to look better."

For an interior designer, those sentences contain quite a lot of useful information.

For an image generation model, they are ambiguous.

I kept running into that problem while building AI Room Design.

At first, I assumed the difficult part would be generating beautiful rooms.

That was wrong.

Modern image models can already generate attractive rooms.

The harder problem was turning vague human taste into enough structure and constraints to make the result useful.

It also changed how I think about consumer AI products.


Users don't speak in prompts

As developers, it is easy to build an AI interface like this:

Describe your dream room:

[________________________________]

                  Generate
Enter fullscreen mode Exit fullscreen mode

Technically, it works.

The model gets maximum flexibility, and the frontend stays simple.

But it also hands the hardest part of the product back to the user.

The user now has to know how to write something like:

Warm neutral color palette,
light oak furniture,
low-profile seating,
natural linen upholstery,
soft indirect lighting,
limewashed walls,
minimal decoration,
subtle Japanese influences...
Enter fullscreen mode Exit fullscreen mode

Most people do not describe rooms that way.

They describe what they want.

Make it warmer.

Make it feel more expensive.

I like Scandinavian interiors,
but I don't want it to feel cold.

Keep the sofa.

I want less clutter.

Make it cozy but still modern.
Enter fullscreen mode Exit fullscreen mode

The user should not have to learn prompt engineering before the product becomes useful.

That led to a principle I now use when designing consumer AI tools:

Don't make users write better prompts. Build the prompting into the product.


A style is more than a string

One way to simplify the experience is to give users style presets.

A side-by-side comparison between a blank AI prompt box and a structured room-design interface with room type, style, goal, and preservation options.

Instead of asking them to describe everything, let them choose:

Japandi
Warm Minimalist
Organic Modern
Mid-Century Modern
Modern Luxury
Industrial
Enter fullscreen mode Exit fullscreen mode

At first, it's tempting to treat these as labels that simply go into a prompt.

Something like:

Redesign this living room in Japandi style.
Enter fullscreen mode Exit fullscreen mode

But "Japandi" is not a complete instruction.

A useful style preset is closer to a bundle of design constraints:

{
  "style": "Japandi",
  "materials": [
    "light natural wood",
    "linen",
    "stone",
    "matte ceramics"
  ],
  "palette": [
    "warm white",
    "beige",
    "muted brown"
  ],
  "furniture": {
    "profile": "low",
    "shape": "simple",
    "decoration": "restrained"
  },
  "lighting": "soft natural light",
  "mood": [
    "calm",
    "warm",
    "minimal"
  ]
}
Enter fullscreen mode Exit fullscreen mode

The user only needs to know one word:

Japandi.

The product turns that word into something the generation pipeline can use.

This is one way consumer AI interfaces differ from general-purpose AI chat interfaces.

The visible control can stay simple.

The instruction behind it does not have to be.


Presets are not enough either

Presets create an opposite problem.

If everything is a preset, the product starts to feel rigid.

Real users don't always want:

"Give me Japandi."

They want:

"Give me Japandi, but keep my current sofa."

Or:

"Make it more modern, but I don't want white walls."

Or:

"I want something between Scandinavian and a luxury hotel."

A fixed style picker cannot express all of that.

A completely open prompt box can express it, but it asks too much of the user.

I started thinking about the interface as a combination of:

Structured controls
+
Optional natural language
Enter fullscreen mode Exit fullscreen mode

For example:

Room
→ Living Room

Style
→ Japandi

Main goal
→ Warmer

Keep
→ Existing sofa

Additional request
→ Add more hidden storage
Enter fullscreen mode Exit fullscreen mode

The application can turn those choices into a richer generation instruction.

I find this more useful than putting ChatGPT-style text input into every AI product.

The user expresses what they want.

The software turns that intent into a prompt.


Prompt writing vs. prompt compilation

This is the mental model I prefer:

Prompt compilation instead of prompt writing.

In a general-purpose AI tool, the user writes the prompt.

In a specialized AI product, the product should handle much of that work.

The pipeline looks more like this:

User intent
     ↓
UI controls
     ↓
Structured parameters
     ↓
Prompt compilation
     ↓
Model instructions
     ↓
Generation
Enter fullscreen mode Exit fullscreen mode

This also makes the product easier to improve.

If users control a giant prompt box directly, every request can look completely different.

If the product first converts intent into structured information, you can reason about it more systematically.

For example:

{
  "room_type": "living_room",
  "primary_style": "japandi",
  "warmth": "warm",
  "preserve": [
    "sofa",
    "windows",
    "doors"
  ],
  "change": [
    "lighting",
    "wall_finish",
    "decor",
    "storage"
  ],
  "custom_request": "avoid pure white walls"
}
Enter fullscreen mode Exit fullscreen mode

Now the generation layer receives more predictable input.

That matters when you are building a product rather than a demo.


Beautiful does not mean useful

Another problem with AI interior design became clear quickly.

Suppose a model produces an image with:

  • beautiful lighting
  • expensive-looking furniture
  • perfect styling
  • magazine-quality composition

It looks great.

Then you compare it with the original room.

The window has moved.

The door has disappeared.

The room is somehow 30% larger.

The ceiling height has changed.

A wall has appeared where no wall exists.

From an image generation perspective, the result may be excellent.

For interior visualization, it is almost useless.

The distinction is straightforward:

Inspiration image ≠ Visualization
Enter fullscreen mode Exit fullscreen mode

If someone wants interior design inspiration, the model can be very creative.

If someone is considering how to redesign their actual room, creativity needs boundaries.

That led to another principle:

The closer an AI output gets to a real purchasing decision, the less freedom the model should have.

For a fantasy room on Pinterest, moving a window does not matter.

For someone deciding whether a new sofa will work in their living room, it matters.


The original room is part of the prompt

The uploaded photo is more than a visual reference.

It contains constraints, including:

Room proportions

Window positions

Door positions

Perspective

Major architectural structures

Existing furniture the user wants to keep
Enter fullscreen mode Exit fullscreen mode

The generation model should not simply ask:

"What would a beautiful Japandi living room look like?"

It should instead ask:

"What would this particular room look like if its design direction changed to Japandi?"

Those are different tasks, and the second is much harder.


Why one generation is not enough

People redesigning a room are rarely looking for one objectively correct answer. They are comparing possibilities.

Someone may like all of these:

Japandi

Warm Minimalist

Organic Modern

Mid-Century Modern
Enter fullscreen mode Exit fullscreen mode

But they do not know which one will look best in their actual space.

That makes AI interior design well suited to comparison.

Rather than:

Generate
   ↓
Like it?

No
   ↓
Regenerate
Enter fullscreen mode Exit fullscreen mode

I think about the workflow this way:

Original room
       ↓
┌──────────────┐
│   Japandi    │
├──────────────┤
│ Warm Minimal │
├──────────────┤
│Organic Modern│
├──────────────┤
│ Mid-Century  │
└──────────────┘
       ↓
Compare
       ↓
Decide
Enter fullscreen mode Exit fullscreen mode

Alongside creating content, the model helps the user explore a decision space.

The difference is easy to miss, but it shapes the product.


AI interior design is a comparison problem

This changed how I think about the final result screen.

The usual AI product flow is:

Input → Output
Enter fullscreen mode Exit fullscreen mode

But many real-world decisions look more like this:

Input
  ↓
Multiple plausible alternatives
  ↓
Comparison
  ↓
Preference
  ↓
Decision
Enter fullscreen mode Exit fullscreen mode

Interior design fits this pattern.

A user may discover:

"I thought I wanted Scandinavian, but the warmer Japandi version actually fits the room better."

That is useful even if they never recreate the generated room exactly.

The AI can surface a preference that was hard to articulate.


Localization is more than translation

Building an international product introduced another problem I did not appreciate at first.

It is easy to assume localization looks like this:

Japandi
→ ジャパンディ

Modern
→ モダン

Natural
→ ナチュラル
Enter fullscreen mode Exit fullscreen mode

But interior design vocabulary is not always that simple.

The visual expectations behind a style can vary by market.

A user searching for a "Natural" interior in Japan may have a different default aesthetic in mind than someone searching for a "Natural" interior in the United States.

The same is true for terms like:

Modern

Luxury

Country

Mediterranean

Minimalist
Enter fullscreen mode Exit fullscreen mode

They can carry different visual associations depending on the country.

Localization may eventually look more like:

Language
+
Local design vocabulary
+
Local housing characteristics
+
Popular materials
+
Current aesthetic preferences
        ↓
Localized style definition
Enter fullscreen mode Exit fullscreen mode

That goes far beyond translating button labels.

It becomes more important as AI products move from generic generation toward specialized domains.


Specialized AI products need opinions

General AI tools are powerful because they're flexible.

Vertical AI products become useful partly because they're not completely flexible.

They encode opinions about the task.

For example:

This is what Japandi usually means.

These materials typically fit the style.

These colors tend to work together.

These parts of the original room should remain unchanged.

These user choices matter more than others.
Enter fullscreen mode Exit fullscreen mode

Each decision gives the model less freedom. In return, the user has less work to do.

That is the trade-off.

A useful specialized AI product often has to make more decisions for the user than a general-purpose chatbot.


My current mental model

The workflow I have in mind looks roughly like this:

Room Photo
    ↓
Understand the Existing Space
    ↓
Capture User Intent
    ↓
Structure the Intent
    ↓
Apply Style Constraints
    ↓
Preserve Important Room Features
    ↓
Compile Generation Instructions
    ↓
Generate Multiple Alternatives
    ↓
Compare
    ↓
Make a Decision
Enter fullscreen mode Exit fullscreen mode

What stands out is how little of this diagram is simply:

Call image generation API
Enter fullscreen mode Exit fullscreen mode

The model is important, but much of the product value comes from everything around it.


The model is not the product

When a new image model launches, it's easy to focus on benchmarks:

Better prompt adherence.

Better photorealism.

Higher resolution.

Better image editing.

Those improvements matter.

But they're ingredients.

A consumer product also needs:

Model capability
×
Input quality
×
Constraints
×
Consistency
×
Interaction design
×
Domain knowledge
×
Trust
Enter fullscreen mode Exit fullscreen mode

If one part is weak, the best available model doesn't necessarily create the best product.

That has been one of the main lessons I have taken from building AI products recently.


A question I now ask about new AI models

Whenever a new image generation model appears, the obvious question is:

"What can this model generate?"

I now ask a different question:

What real-world decision was previously too difficult for software, but has become practical because of this model?

Interior design is one example.

Before generative image models, it was expensive to show someone multiple realistic redesigns of their exact room.

You needed professional software, design skills, rendering skills, or an interior designer.

Now the cost of exploring a visual possibility is approaching zero.

That changes the product opportunity.

What changes is not simply this:

AI can generate rooms.

It's this:

People can explore expensive real-world decisions before spending money on them.

Furniture.

Paint colors.

Room styles.

Layouts.

Materials.

Renovations.

That's the part I find more interesting.


What I would do differently if I started again

If I were rebuilding the product from scratch, I would focus much earlier on the interface between human intent and model instructions.

I would spend less time asking:

"How do I create the perfect prompt?"

And more time asking:

"What does the user actually know about what they want?"

They often know more than they think:

I like this.

I hate that.

Keep this sofa.

More storage.

Less gray.

Warmer.

Brighter.

Not too minimal.
Enter fullscreen mode Exit fullscreen mode

The product's job is to turn those fragments into something the model can use.

That is not really a prompt engineering problem.

It is a product design problem.


Where the project is now

I eventually turned these ideas into AI Room Design. I am experimenting with letting people upload photos of their existing spaces, explore different interior styles, add their design preferences, and compare redesign directions.

I'm still learning where the right boundary is between:

AI creativity
Enter fullscreen mode Exit fullscreen mode

and:

Real-world constraints
Enter fullscreen mode Exit fullscreen mode

Too much constraint makes every result look predictable.

Too little constraint produces beautiful rooms that have nothing to do with the space the user owns.

Finding that balance has been more interesting than simply connecting an image model to an upload button.

It has also left me convinced of one thing:

The future of consumer AI interfaces probably isn't better prompt boxes.

It's software that understands what users are trying to accomplish well enough that they don't need to think about prompting at all.

Top comments (0)