DEV Community: Pavel

Vibe Game Engine: The Road to AI-Editable Game Dev 🚀

Pavel — Sun, 11 May 2025 19:58:55 +0000

AI Chaos & Experiment:

When I started building an engine with an AI assistant, I thought - it would be cool, fast, convenient.
But it turned out to be a constant experiment where I never know what will happen tomorrow.
Sometimes AI generates brilliant ideas, sometimes it goes off the rails, and I sit there thinking: can you even build something truly usable with it?
Sometimes it feels like I'm not coding, but managing the chaos my own AI partner throws at me.
This article is an honest experiment diary.
No guarantee of success, but full of vibe and surprises. 🎲

Now - how I got here, and why AI is even involved... 🤖

The Birth of an Idea: "Make a Simulator of Myself? Why Not!" 💡

It all started with a sudden thought: "Why not make a programmer life simulator using a new approach - Vibecoding?" 💭

I wrote the first lines of code in seconds, and suddenly the game was almost ready.
The first "player" appeared, his sprite - and the outlines of a future virtual world started to take shape.
But that was just the beginning of my journey into the world of "vibecoding".
The idea seemed so original and meta-cool that the decision came by itself: for a game created in such an unusual way, you need your own unique engine.
Modern engines aren't suitable for working with AI assistants: their internals are closed, and AI can't fully control or change anything inside.
So I wanted to try making something of my own - open for experiments and true AI collaboration. 🛠

Vibecoding in Action: When AI Is Your Co-Author (Sometimes Very Stubborn) 🤝

An essential part of "vibecoding" is its unpredictability.
Sometimes, the AI assistant enthusiastically generates hundreds of lines of code for a new feature, but... it just doesn't work.
At all.
So I sit there, analyzing this monumental labor of artificial intelligence, trying to find the bug.
Then, after long reflection (and maybe more than one cup of coffee ☕), I spot some silly detail: a variable declared in the wrong scope, or one line of code conflicting with another.
I move that cursed variable to the right scope - and, miracle, everything magically comes to life!
In those moments, you feel like both a detective 🕵 and a bit of a wizard 🧙, realizing that even the most advanced AI can't fully replace human intuition and experience.
But this iterative debugging and problem-solving process together is the real "vibe". ✨

Sometimes arguing with AI becomes a sport of its own: it gets lazy, acts dumb, you want to curse at it, and some models (I think it was gemeni) can even sass you back.
At times like these, you realize working with AI isn't just about code - it's about nerves, patience, and a sense of humor. 😅

What annoys me most is when the model does something you didn't expect, you point it out, and it replies: "I did as you asked" and even quotes your own words.
That's when you really want to turn off the computer and go drink vodka. 🍵

I work in parallel with all the latest models: sonnet 3.7, gemeni 2.5, gpt 4.1.
And honestly, only sonnet and gemeni let you actually write code and get sensible answers - the rest either do nothing or try to build a parallel architecture for a tiny feature. 🤦

Sometimes you spend hours trying to "persuade" AI to generate something that works, editing, regenerating, editing again...
And at some point you realize it's more efficient to just hit "reset", delete all the AI's creative output, and start over.
Or, with a deep sigh, just sit down and write that piece of code yourself, old-school.
That's also "vibecoding" - mental flexibility and knowing when to trust the machine, and when to trust your own hands and head. 💪

What Came Out of It

The transformation system - the foundation of positioning, rotation, and scaling objects - went through many iterations. While this is standard functionality for any engine, building it together with AI was unique in its own way. Modern models don't understand the concept of space. At that moment, the illusion of intelligence collapses: you realize it's just a token sequence generator - formally correct, but with no idea how it exists in the real world. In the end, I managed to make components responsible for visualization (whether static images or animated sprites) automatically sync with transformation changes, ensuring system responsiveness. 🖼
Then came the turn for the physics system - again, no understanding of world concepts or how to connect it to transformations. After a day of "trial and error", I just sat down and wrote it myself. ⚡
And what's an engine without its own editor? Me and AI created a set of tools to make the development process as comfortable as possible - here, AI was actually great, though I had to constantly nudge it to reuse existing code instead of rewriting all the styles from scratch every time:
Object Inspector: Lets you deeply inspect any game object, view its components (like transform or sprite), and quickly tweak their properties. New components can be added "on the fly" or unnecessary ones removed.
Asset Manager: A handy panel for managing graphics and other project resources. Assets can be easily dragged right onto the game scene.
Sprite Sheet Editor: My special pride! A separate window where you can upload an image (sprite sheet) and visually "slice" it into frames for animation. You can create frames, change their size and order, and save this info. The sprite component then uses this data for smooth animation. No more tedious manual frame coordinate setup!
Gizmo: An interactive tool on the scene for visually manipulating selected objects: move, rotate, scale. I paid special attention to making Gizmo intuitive and working correctly with the object's pivot point. You can probably guess - the model had no idea how to make this work. But after the 10th regeneration, it more or less worked. 🛠

So What's the Result? An Engine Born of Vibe and Code! 🎮

The result is not just a pile of code, but a full-fledged game engine!
Yeah, maybe it's not perfect, and its history has moments that make you smile, but it's mine, unique, built on pure enthusiasm, a flow of inspiration, and close collaboration with an AI assistant.

This engine is living proof that "vibecoding" with AI isn't just a buzzword, but a whole development philosophy.
It lets you not be afraid of experiments, boldly face mistakes (they're part of the journey!), and move forward with confidence.
It's a process where it's not scary to try new things, even if the intermediate results sometimes look... eccentric.

The main thing is the creative search itself, that unique "vibe" from creating something new, something yours, together with AI.
And this engine is the brightest proof of that. ✨

No-Drama Finale:
So that's the kind of vibe engine I ended up with.
Not perfect, not by the book, but mine - with bugs, memes, and a ton of pain to share with friends over pizza. 🍕

And yeah, not a single system is 100% finished: there are bugs and logic dead ends everywhere, and fixing them means it's easier to just rewrite everything from scratch.
Siccing an agent on refactoring isn't a great idea either: it'll run out of context or just lose the "thread of reasoning". 🤷

If you've read this far - respect!
Don't be afraid to experiment, argue with AI, sometimes wipe everything and start over.
That's the real joy of vibecoding.
And if you've had your own epic or funny moments with AI - be sure to share them in the comments.
Let's collect a gallery of vibey stories and laugh together! 😂

Tired of Fake Code Assistants... 😠🌈

Pavel — Tue, 28 Mar 2023 10:39:00 +0000

I've tried countless code assistants, but most of them just seem to slap ChatGPT into my code editor and call it a day.
They claim to be "revolutionary" and "life-changing," but in reality, I could just use ChatGPT by myself. 🙂

How a real code assistant should work: You share your coding needs, and - bam - it whips up flawless code without breaking a sweat!

Oh, you need a function? Sure, AltQ will whip one up just for you. And wait, you want a code change? Well, of course, AltQ will cater to your every whim and modify the code exactly as you desire. Because that's what a real code assistant should do, right? 😉

Well, here's the link, folks! You know what to do.

Fear Not the Rise of AI: Meet AltQ, Your New Coding Companion, and Unlock Endless Opportunities!

Pavel — Mon, 27 Mar 2023 09:02:29 +0000

Meet AltQ, Your New Coding Buddy

Introducing AltQ, the code assistant so cool, it'll make you question if you even need coffee to code anymore.
It's a Visual Code extension that seamlessly integrates with your coding environment.

Picture this: You tell AltQ what you need, and
poof – it spits out the code like it's no big deal!

AltQ is like a programmer's dream come true – it understands human lingo and turns it into code.

AltQ ensures a smooth coding experience, making it feel like you have your own dedicated protégé ready to execute any task you assign.

AltQ in Action: Creating a Tank Game Together Like a Coding Dream Team

Get ready to be amazed, because AltQ and I just joined forces to create a fantastic little tank game, complete with user controls, enemy AI, game states, and a minimalistic interface!

But wait, there's more: we didn't just stop at building the game elements, we also whipped up a simple game engine to bring our tank-tastic vision to life.

It was like having a coding sidekick who knew exactly what I wanted, anticipating my needs and helping me blast through obstacles like a well-aimed tank shot.

With AltQ's guidance and support, we breezed through the development process, proving that when it comes to coding, two heads (or one human and one AI) are definitely better than one!

AltQ vs. Copilot and Rest: A Unique Approach to Code Assistance.

Other code assistants like Copilot and ChatGPT-integrated Visual Code have their charm, but AltQ brings a unique flavor to the coding fiesta.

AltQ isn't just about offering snippets or chatting; it's about creating a code assistant that truly understands your project and your goals.

The key differentiator: AltQ dives deep into the context of your work, grasping the essence of your project and tackling routine problems with ease.

Ready, Set, AltQ: A Quick Guide to Installation and Setup

Eager to experience the awesomeness of AltQ? Here's your handy-dandy guide to getting it installed and configured for Visual Code. It's like setting up a playdate with your new AI buddy!

Step 1: Head over to the Visual Code Marketplace and search for the AltQ extension. Once you find it, click the "Install" button to add it to your coding arsenal.

Step 2: To make the magic happen, you'll need an API key from OpenAI. Visit the OpenAI website, sign up or log in, and get your hands on that precious key.

Step 3: With your API key in hand, it's time to configure AltQ. You can do this in one of two ways:

Option A: Dive into the settings of Visual Code, locate the AltQ extension settings, and enter your API key.
Option B: If you're more of a command prompt kind of coder, you can also enter your API key the first time you run the AltQ prompt command.

And that's it! Simply hold the Alt key + Q, and you're all set to start your coding adventure with AltQ, your trusty AI sidekick. Happy programming!

Can't Beat AI? Join 'em, Lead 'em, and Rock On!

Addressing the elephant in the room: the fear of job loss due to AI (like AltQ) stealing the coding thunder.

No need to panic! The focus shifts from manual coding to strategic thinking and problem-solving – it's time to level up.

Embrace the change, hone new skills, and become a trailblazer in the AI-powered world of programming.

Highlighting one of AltQ's killer features: outsourcing memory-intensive tasks to AI, so you can focus on being creative.
Human memory has limits, but with code assistants like AltQ, you can break those boundaries and reach new heights in programming.
Highlighting one of AltQ's killer features: outsourcing memory-intensive tasks to AI, so you can focus on being creative.
Human memory has limits, but with code assistants like AltQ, you can break those boundaries and reach new heights in programming.
Let AI handle the details and remember countless entities, while you bring your A-game to the table and innovate like never before.

AI assistants changing the game, and it's time to jump on board the coding revolution!

AltQ Alpha: A Work in Progress with Big Dreams and Plans

Before you dive headfirst into the world of AltQ, let's take a moment to remember that it's still in its early alpha stage. Like a newborn kitten finding its footing, AltQ has some growing to do.

Currently, AltQ relies on the GPT-3 model to generate answers. While it's quite impressive, it's a bit outdated and might require some patience and persistence to achieve the desired results.

But fear not, fellow coding enthusiasts! I've got grand plans to refine the engine and improve the entire process – this is just the first step on the exciting journey.

So, buckle up and join me as we help AltQ evolve from an alpha version into the game-changing code assistant!

Awaiting GPT-4: AltQ's Bright Future and Our Collective Efforts

While the current GPT-3.5 API isn't quite the perfect fit for AltQ, the future is looking bright! I've had the chance to tinker with GPT-4, and let me tell you – it's downright incredible.

Unfortunately, my GPT-4 API access is still on the waitlist.
Let's make some noise! Together, we can raise awareness about AltQ, and with a little buzz, we might just get that API access sooner rather than later.

So, spread the word, share your excitement, and let's help AltQ reach its full potential with the power of GPT-4. The future of AI-driven code assistance awaits!

The Future of AltQ: A Code Assistant That Knows Your Project Inside Out

Picture this: AltQ becomes your go-to companion, fully understanding your project, knowing the ins and outs of each line of code, and grasping the impact of every variable.

With its incredible AI capabilities, AltQ will create required features and project artifacts on your behalf, making your coding life a whole lot easier.

But it doesn't stop there – AltQ will free up your mental bandwidth, allowing you to focus on tackling the truly complex challenges and making groundbreaking discoveries.

Together, let's turn this vision into reality and shape the future of AI-driven code assistance with AltQ as our trusty sidekick – not a job-stealing "Skynet", but a helpful companion for all developers.

AltQ

AltQ: GPT Power in Exploring the Possibilities of a Personal Code Assistant

Pavel — Sun, 26 Mar 2023 09:18:26 +0000

Meet AltQ, Your New Coding Buddy

Introducing AltQ, the code assistant so cool, it'll make you question if you even need coffee to code anymore.
It's a Visual Code extension that seamlessly integrates with your coding environment.

Picture this: You tell AltQ what you need, and
poof – it spits out the code like it's no big deal!

AltQ is like a programmer's dream come true – it understands human lingo and turns it into code.

AltQ ensures a smooth coding experience, making it feel like you have your own dedicated protégé ready to execute any task you assign.

AltQ in Action: Creating a Tank Game Together Like a Coding Dream Team

Get ready to be amazed, because AltQ and I just joined forces to create a fantastic little tank game, complete with user controls, enemy AI, game states, and a minimalistic interface!

But wait, there's more: we didn't just stop at building the game elements, we also whipped up a simple game engine to bring our tank-tastic vision to life.

It was like having a coding sidekick who knew exactly what I wanted, anticipating my needs and helping me blast through obstacles like a well-aimed tank shot.

With AltQ's guidance and support, we breezed through the development process, proving that when it comes to coding, two heads (or one human and one AI) are definitely better than one!

AltQ vs. Copilot and Rest: A Unique Approach to Code Assistance.

Other code assistants like Copilot and ChatGPT-integrated Visual Code have their charm, but AltQ brings a unique flavor to the coding fiesta.

AltQ isn't just about offering snippets or chatting; it's about creating a code assistant that truly understands your project and your goals.

The key differentiator: AltQ dives deep into the context of your work, grasping the essence of your project and tackling routine problems with ease.

Ready, Set, AltQ: A Quick Guide to Installation and Setup

Eager to experience the awesomeness of AltQ? Here's your handy-dandy guide to getting it installed and configured for Visual Code. It's like setting up a playdate with your new AI buddy!

Step 1: Head over to the Visual Code Marketplace and search for the AltQ extension. Once you find it, click the "Install" button to add it to your coding arsenal.

Step 2: To make the magic happen, you'll need an API key from OpenAI. Visit the OpenAI website, sign up or log in, and get your hands on that precious key.

Step 3: With your API key in hand, it's time to configure AltQ. You can do this in one of two ways:

Option A: Dive into the settings of Visual Code, locate the AltQ extension settings, and enter your API key.
Option B: If you're more of a command prompt kind of coder, you can also enter your API key the first time you run the AltQ prompt command.

And that's it! Simply hold the Alt key + Q, and you're all set to start your coding adventure with AltQ, your trusty AI sidekick. Happy programming!

Can't Beat AI? Join 'em, Lead 'em, and Rock On!

Addressing the elephant in the room: the fear of job loss due to AI (like AltQ) stealing the coding thunder.

No need to panic! The focus shifts from manual coding to strategic thinking and problem-solving – it's time to level up.

Embrace the change, hone new skills, and become a trailblazer in the AI-powered world of programming.

AI assistants changing the game, and it's time to jump on board the coding revolution!

AltQ Alpha: A Work in Progress with Big Dreams and Plans

Before you dive headfirst into the world of AltQ, let's take a moment to remember that it's still in its early alpha stage. Like a newborn kitten finding its footing, AltQ has some growing to do.

Currently, AltQ relies on the GPT-3 model to generate answers. While it's quite impressive, it's a bit outdated and might require some patience and persistence to achieve the desired results.

But fear not, fellow coding enthusiasts! I've got grand plans to refine the engine and improve the entire process – this is just the first step on the exciting journey.

So, buckle up and join me as we help AltQ evolve from an alpha version into the game-changing code assistant!

Awaiting GPT-4: AltQ's Bright Future and Our Collective Efforts

While the current GPT-3.5 API isn't quite the perfect fit for AltQ, the future is looking bright! I've had the chance to tinker with GPT-4, and let me tell you – it's downright incredible.

So, spread the word, share your excitement, and let's help AltQ reach its full potential with the power of GPT-4. The future of AI-driven code assistance awaits!

The Future of AltQ: A Code Assistant That Knows Your Project Inside Out

Picture this: AltQ becomes your go-to companion, fully understanding your project, knowing the ins and outs of each line of code, and grasping the impact of every variable.

With its incredible AI capabilities, AltQ will create required features and project artifacts on your behalf, making your coding life a whole lot easier.

But it doesn't stop there – AltQ will free up your mental bandwidth, allowing you to focus on tackling the truly complex challenges and making groundbreaking discoveries.

AltQ

AltQ Alpha: Exploring the Possibilities of a Personal Code Assistant

Pavel — Sat, 25 Mar 2023 23:12:00 +0000

Meet AltQ, Your New Coding Buddy

Introducing AltQ, the code assistant so cool, it'll make you question if you even need coffee to code anymore.
It's a Visual Code extension that seamlessly integrates with your coding environment.

Picture this: You tell AltQ what you need, and
poof – it spits out the code like it's no big deal!

AltQ is like a programmer's dream come true – it understands human lingo and turns it into code.

AltQ ensures a smooth coding experience, making it feel like you have your own dedicated protégé ready to execute any task you assign.

AltQ in Action: Creating a Tank Game Together Like a Coding Dream Team

Get ready to be amazed, because AltQ and I just joined forces to create a fantastic little tank game, complete with user controls, enemy AI, game states, and a minimalistic interface!

But wait, there's more: we didn't just stop at building the game elements, we also whipped up a simple game engine to bring our tank-tastic vision to life.

It was like having a coding sidekick who knew exactly what I wanted, anticipating my needs and helping me blast through obstacles like a well-aimed tank shot.

With AltQ's guidance and support, we breezed through the development process, proving that when it comes to coding, two heads (or one human and one AI) are definitely better than one!

AltQ vs. Copilot and Rest: A Unique Approach to Code Assistance.

Other code assistants like Copilot and ChatGPT-integrated Visual Code have their charm, but AltQ brings a unique flavor to the coding fiesta.

AltQ isn't just about offering snippets or chatting; it's about creating a code assistant that truly understands your project and your goals.

The key differentiator: AltQ dives deep into the context of your work, grasping the essence of your project and tackling routine problems with ease.

Ready, Set, AltQ: A Quick Guide to Installation and Setup

Eager to experience the awesomeness of AltQ? Here's your handy-dandy guide to getting it installed and configured for Visual Code. It's like setting up a playdate with your new AI buddy!

Step 1: Head over to the Visual Code Marketplace and search for the AltQ extension. Once you find it, click the "Install" button to add it to your coding arsenal.

Step 2: To make the magic happen, you'll need an API key from OpenAI. Visit the OpenAI website, sign up or log in, and get your hands on that precious key.

Step 3: With your API key in hand, it's time to configure AltQ. You can do this in one of two ways:

Option A: Dive into the settings of Visual Code, locate the AltQ extension settings, and enter your API key.
Option B: If you're more of a command prompt kind of coder, you can also enter your API key the first time you run the AltQ prompt command.

And that's it! Simply hold the Alt key + Q, and you're all set to start your coding adventure with AltQ, your trusty AI sidekick. Happy programming!

Can't Beat AI? Join 'em, Lead 'em, and Rock On!

Addressing the elephant in the room: the fear of job loss due to AI (like AltQ) stealing the coding thunder.

No need to panic! The focus shifts from manual coding to strategic thinking and problem-solving – it's time to level up.

Embrace the change, hone new skills, and become a trailblazer in the AI-powered world of programming.

AI assistants changing the game, and it's time to jump on board the coding revolution!

AltQ Alpha: A Work in Progress with Big Dreams and Plans

Before you dive headfirst into the world of AltQ, let's take a moment to remember that it's still in its early alpha stage. Like a newborn kitten finding its footing, AltQ has some growing to do.

Currently, AltQ relies on the GPT-3 model to generate answers. While it's quite impressive, it's a bit outdated and might require some patience and persistence to achieve the desired results.

But fear not, fellow coding enthusiasts! I've got grand plans to refine the engine and improve the entire process – this is just the first step on the exciting journey.

So, buckle up and join me as we help AltQ evolve from an alpha version into the game-changing code assistant!

Awaiting GPT-4: AltQ's Bright Future and Our Collective Efforts

While the current GPT-3.5 API isn't quite the perfect fit for AltQ, the future is looking bright! I've had the chance to tinker with GPT-4, and let me tell you – it's downright incredible.

So, spread the word, share your excitement, and let's help AltQ reach its full potential with the power of GPT-4. The future of AI-driven code assistance awaits!

The Future of AltQ: A Code Assistant That Knows Your Project Inside Out

Picture this: AltQ becomes your go-to companion, fully understanding your project, knowing the ins and outs of each line of code, and grasping the impact of every variable.

With its incredible AI capabilities, AltQ will create required features and project artifacts on your behalf, making your coding life a whole lot easier.

But it doesn't stop there – AltQ will free up your mental bandwidth, allowing you to focus on tackling the truly complex challenges and making groundbreaking discoveries.

AltQ

Layout to HTML and back

Pavel — Tue, 25 Aug 2020 19:49:04 +0000

How to unify an engineer and GUI designer? How do I make them work with same entities?
How to keep mockups in a version control system?
If you are interested in those questions, welcome to aboard!

Before start

This post is a sequel of my previous post "No more HTML coding, please. Let's make layout to HTML real" and the result of research interesting me topic.

This is my second English publication, please be lenient. Any fix suggestions are welcome.

In the previous post, I briefly described current problems of the HTML code automatization and shared my research results. I stopped at the control problem.
In this post I will discover this concept in more detail and share the results of my research.
Want to touch something? Here is the proof of concept.

The Control Problem

I defined two controlling directions:

The possibility to affect to process of HTML code generation.
Direct editing the result which will be reflected in the next generation.

As a part of my research I looked at the first direction and came to the conclusion that the HTML generation should be built on a declarative language of rules. This approach will give a high level of control on a resulting code and opportunities to make own rules set to developers.
But now I would like to dealt with in greater detail on the second direction.

Many of graphical web editors offer HTML code export but nothing more. This HTML code is derived and any changes on it will be lost after the next generating.

Thus, building the interface and applications on this approach is very difficult. Not to mention the quality of the generated code. This is quite enough for the tasks being solved for this services such as landing page generators or prototyping, but for applications, the best they can do is be a quick start for something more complex.

The solution

The main idea is that if layouts are made to create an HTML document, then the entire layout can be described in HTML and CSS.
Based on this, we can use the HTML document as a source of truth, without the need for a third format for storing layouts.

But if the layout is described in an HTML document, then what prevents the developer from opening the code and changing it in any way he wants. The answer is nothing, yes with reservations, but it is possible.
This way we have a 2-way binding between the design tool and the source code.

Perhaps someone will say that this sounds like a utopia, Unity 3D, I will say back.
Unity, like any other game engine, has implemented a similar concept in their tools for a long time.

In my opinion, the only obstacle to implementing such an approach in the web industry and derivatives is the high requirements for HTML code, I looked up this issue in more detail in a previous article.

The only thing that I will add. Continuing the comparison with game engines, is that in addition to the tree of the hierarchy of objects that the specialist works with, the engine itself builds many other trees that no one is interested in knowing about. Unlike the web.

Ok, I want to have something like Unreal Engine, but for the WEB. Where should I start?
Right now I have:

A simple Drag&Drop editor;
A concept of auto-coding module;
Some of HTML code that I want to edit.

The first question is how to put it all together?

The Engine

Any engine consists of modules, so it's worth starting with this.

It's hard not to start to imagine a system with modules like api mocking, testing, storage management, animation creation, optimization, working with dependencies, documenting ...

But it's worth cool down and return to the current problems.

The Rendering Module

Perhaps the question is obvious, but in general, what are we drawing and where? We have HTML code that we need to display with something.
The logical choice here would be the most popular render - ~~WebKit~~ Blink, we will use it as a rendering module.

The Read-write Module

But with the reader, everything is not so obvious. As long as we are talking about a single entry point (all the code is in index.html), there are no problems. But a real application can have an arbitrarily complex architecture and can be stored in any format.
I have identified 3 options for how to deal with it:

Working with the final code, with intermediate rendering;
Maximum coverage of all popular technology stacks;
Strict standardization.

There is no better option, each has its drawbacks.

The most reliable seems to be the third, the most convenient for work is the second. I suppose that a mixed approach with a gradual adding of the stacks would be optimal, and where this is impossible, fallback to the third.

The first option seems to be very promising, but with a number of difficult issues, the solution of which may take longer than the second option.

Of the obvious problems:

Comparison of the intermediate to the original.
Converting the intermediate code to the original format.

The problems are interesting and require research.

The Auto HTML Code Module

Probably the easiest part.

The Visual Interaction Module

And what can we exactly interact with?

This is what a tree looks like on one of the landing pages, describing 2 blocks lying side by side. If it is justified there, then this is completely inappropriate in the interface editor.

Looking at it, I recall Adobe Illustrator in which, to get to the desired element, you need to click on the same place 2 * N times.

But if we look closely, the editable elements there are only the heading and paragraph. The rest describe the arrangement of the elements.

This fact is well illustrated the following concept - not all HTML code is the same.

From this we formulate a rule. HTML code is divided into:

Significant;
Insignificant.

An user interacts with significant items.

The significant code describes an appearance, meta and structure: grouping, positioning, etc.

The editor itself must meet all the expectations of the designer.

The Binding

And what do we have and what do we binding with?

Some code as a source of truth;
A DOM tree;
A Visualization.

When we change the code then the DOM is changes. That guides to re-rendering.

When we change the visualization:

Modify DOM, store derived result to code;
Change the code that will change the DOM.

Each of the options will redraw the display.

The state

But what is the lifecycle of the DOM? In addition to the editor, logic can also change it.

This means that we need to determine the current state of the code and select it from the list of all states and edit the changes exactly where it is needed.

The task is additionally complicated by the fact that a new state can completely change the interface, and coupled with an almost endless combinations of options for changing states. This task seems to be completely solvable only at the level of a semantic apparatus comparable to the human one (and the human does not always cope).

All of the above sounds like it’s time to wait for a strong AI, but for now leave this occupation... Or introduce restrictions.

Let's look at the comparison with game engines again. There are always at least 2 modes:

Edit;
Preview.

In the edit mode, interaction only with blocks is possible, let's call them components, with the default state.

As for the rest, I would divide them into 2 types of states:

Manual, those that an editor can understand and give control over them;
Automatic, unclear to an editor, programmed.

A great example of manual ones would be Pagedraw with its state editor and transition between them.

But still, this topic is one of the most difficult, and requires much deeper and more thorough research.

Summary

From the above, the following concept is formed:
We can interact in edit mode with blocks that are significant. These blocks can form components, which in turn can have simple or complex states.
Editing states is a separate task and can be done both visually and through code. Components form a library. All this will impose certain rules and restrictions.

Proof of concept

To proof the concept, it is necessary to implement the editing mode, in which changes in the visual editor affect the code immediately, and changes in the code are also immediately visible in the visual editor.

The code acts as a source of truth and stores representation data, meta information, and structure (child-container).

Subsequent opening of the code restores the state of the visual editor.

Armed with this knowledge, let's implement it in code.

Restrictions

Significant elements are those with a class or identifier;
No side effects applied to items;
An identifier is unique, reuse will lead to unexpected consequences;
Most edge cases are not handled;
Errors are expected;
Components and states not implemented;
Moving a container with children is possible when "grabbing" an empty space.

Conclusion

While 2-way binding is challenging and requires extensive research, it is possible. But it also implies a number of restrictions.

Each of tech stacks will need its own interpreter, which is impossible without the interest of the community of this stack.

What's next?

I have another month of free time, so I have enough playing with "cubes", it's time to implement a clean model of the editor. To begin with, I will use React as the target stack, or rather JSX + styled-components, as the most used and studied.

This process can take quite a long time, realizing this, I brushed the dust off my twitter account. I invite everyone interested to follow the process.
There I will post my thoughts about the problem and the current state of the work, and if the stars will align, then about the result - a free editor for everyone who wants to "move this web".

Of course, I will be very glad to see constructive criticism and discussion.

Peace for everyone!

No more HTML coding, please. Let's make layout to HTML real

Pavel — Sat, 11 Jul 2020 03:22:52 +0000

Do you ever think that handwrote HTML code is expensive, long, redundant and outdated?
I think about it every day, when I create another GUI component or system element. This article is my attempt to overview the current state of the industry, its problems and share the results of my research.

Before we start

Wanna try it right now? Here is the proof of concept.
It shows the concept, it's missing implementation however I think the concept is proven. Details below.

This is my first English publication, please be lenient. Any fix suggestions are welcome.

Glossary

I use "HTML code" and what I mean by this is:

As a HTML development process.
As artifacts obtained after HTML development - HTML, CSS, JS, etc.

Intro

About 8 years ago, I expressed the idea that manual HTML coding is outdated and automation will replace it. From that moment, I have been intently searching for solutions to resolve this issue. We already have such tools as Workflow, Bootstrap Studio, ~~inVision~~, ~~Framer~~, Supernova, React Studio and many other direct or indirect solutions.
And we also have amazing research on this topic using neural networks, via pix2code or sketch2code.

Unfortunately, I can't find a tool that can be fully integrated into my development process.

So what I do I want? I want to get a layout from the layout designer, break it into components, correct the HTML code, where necessary, add logic, get a library of components and return it all to the designer for future interactions. I understand that it surpasses even the most advanced capabilities of the industry, but this is my dream…

Like Confucius said, the long journey begins with the first step, so I decided to figure out where to start. This is what this article will be about.

The Problem

Before all we should to determine the difference between HTML code and interface:

HTML code - is a semi-finished product, roughly speaking, the result of converting a graphic format to HTML, CSS and other files, intended for further conversion to a graphic interface.

Interface - a product ready artifact of the system with which users will interact.

At the 2020, manual HTML coding is still the main way to create web interfaces, which is quite interesting in itself. This goes against the trends of parallel stacks, such as native and desktop applications, where visual interface design tools are the standard for its creation.

We may argue for a long time why this is so, but as I think, the main conclusions are:

high requirements for the final code and
the need for low-level control

I propose to leave the "HTML coders lobby" outside the brackets :).

HTML coding is complex, requires the use of special techniques and tools to manage, minimize errors and support up to date.

Animations are other big question. Creating of complex animation can be impossible task for many companies because it's very laborious.

HTML coding is expensive, on average it's from 25% of the cost of the entire system for SPA and up to 75% for landing pages.

There is no generally accepted HTML coding theory as such.
The W3C standard is very wide and each developer / team is guided by their own rules and standards.

My solution

First We must to formalize the HTML coding process, define the entities, algorithm and rules. Of course, the topic itself is quite extensive, and its full disclosure is not the goal of this article. Based on this, we consider only the part that is currently interest.

Template

Let's try to define process of template creation.

Step 1

Visually divide into a high-level block tree.
Find the rows and columns of the layout.

Step 2

Semantic analysis, we define the main blocks:

Header
Body
Side bar
Footer
Etc…

Here we immediately encounter an interesting phenomenon, a layout is not always enough to build a full-fledged structure, so the interface designer should clarify the behavior of the blocks. Which suggests the idea that semantic analysis is also not always enough.

And for now, this is enough for us. Here we can divide our task into two large groups:

Technical and
Semantic.

Let's set aside the semantic group for future research and focus on the technical one.

Despite the fact that the HTML code of a picture by a neural network is a very interesting task, in my opinion, it's redundant.
It is hard to imagine a situation when, in a normal workflow, an interface designer will send a layout in the format of a bitmap image.

Most often we have formats created in tools such as Figma, Sketch or Adobe Photoshop, which already contain almost exhaustive information about the layout, the most important thing:

Elements position;
Elements type;
Elements styles;

Obtaining an HTML document based on this information is already a solved problem, as engineers from Figma have already shared their conversion implementation and research results, and services such as Anima are directly built on the synchronization of layouts and interface.

But why didn't such decisions lead to the effect of an exploding bomb and after 2 years there is nothing better than the good old hand-made HTML code?
Here I repeat my opinion the reason for this is two factors:

High quality requirements
Need for control

Control is undoubtedly a necessary thing, but without satisfying the first requirement, it does not really make sense. It will be easier to immediately implement and manage a high-quality HTML code than to try to fix a poor-quality one.

Thus, the first requirement is paramount. But what makes a code to be quality?
If we remove official quality indicators such as LTR, Accessibility and the like so, we will remain with important quality indicators for developers:

Correct tree;
Semantics;
Non-redundancy;
Readability and edibility;
Using of blocks taken out of the stream only where it is needed.

Thus, the main task for automation will be compliance with these criteria.

Let's try to prove that this is possible by building a tree of blocks. Again, to do this, we need to formalize the process and introduce the necessary concepts. Leave a number of extreme cases for future research.

Rows and Columns

Rows

If the position of one element falls into the height segment of another, then they form one row.

Columns

Similarly, if the position of one element falls into a segment of the width of another, then they form one column.

Element Relations

Relative layout

Elements are positioned relative to each other, relying on the flow.
The tree is completed if necessary.

Overlapping layout

Overlapping elements break out of the flow, are positioned absolutely, do not affect the positioning of other elements.

The process of building a blocks tree

Step 1

Define all rows.

Step 2

Find margins of each rows.

Step 3

Select a row for work and determine the type of layout:

Single-column;
Multi-column;

Step 4

For multi-column, determine the type of columns:

Floating;
Grid;

Depending on the type, find indent between the columns.

Step 5

Determine the type of column by the number of elements in it:

Single;
Multiple.

If the type is single, position the element relative to the column, go to the next.

Step 6

For the column type "multiple" we find all the rows.
Rows types are similar to column types:

Floating;
Grid;

Find the margins.
We repeat the entire algorithm while there is at least one raw block.

And now we implement the obtained statements into the code.
Simplification:

Fast implementation covering only ~ 20% of cases;
Positioning errors are expected;
One-level structure of source blocks;
Styles are written to the style attribute;

You can see and play with the proof of the concept here.

Conclusion

Automation of high-quality HTML code is possible without involving weakly controlled machine learning algorithms. This will significantly reduce the cost of creating products and increase their quality. It is also important that this will simplify the work of developers and make its more intuitive and enjoyable.

But further research is needed, and the problem itself requires closer attention of the community to implement a fully functional model and tools based on it.

What's next?

I think the next important step is the confirmation of the concept of controllability. The main factor here will be converting the code to the layout. When the code is changed, the layout will also be updated - creating a two-way binding between the design tool and the HTML code.

It will be great to see any constructive criticism and the discussion. Peace!

The next part