DEV Community: Phodal Huang

Architecture Workbench: Building Digital Twins of Enterprise (Application) Architectures

Phodal Huang — Wed, 25 May 2022 11:13:40 +0000

The Architecture Workbench is an environment designed to help people design architectures, evolve architectures, observe architectures, and effectively use the high-quality tools required for architectures, such as interactive architecture development and analysis.

In the previous article "Architecture as Code: Coding Next Generation Enterprise (Application) Architecture Systems", we introduced architecture as code The idea is how to combine and present architectural elements and features around a series of patterns of architecture, and closely integrate architectural decisions and design principles with the system.

In order to implement and implement the concept of architecture as code, it is also necessary to build a platform to run these codes, which we call the architecture workbench. But why are we building an architecture workbench? Just for fun.

Here is screenshots:

Why build Architecture Workbench?

In ArchGuard, we want to manage three forms of architecture: design state, development state and operation state. corresponds to:

Design a new enterprise (application) architecture. Such as describing and designing the current architecture of the system.
Understand and manage the current state of the system. Such as displaying the status quo of the system through visual means and managing the system with rules.
Observe the operation of the system <=> architecture.

Similarly, for the departments related to the structure of many medium and large organizers, they also have the above problems. And, I believe they face the same dilemma that we built ArchGuard:

Architecture is multidimensional. Including technology, data, security, operation and maintenance and systems, etc.
Lack of a unified architecture language. Human language for communication, such as what is a component?
The architecture of the system is very strange. Architectural style or pattern differences, such as microservice architecture, plug-in architecture, etc.
Lack of business context. Being an external architect lacks some context to help with governance.
Details are the devil. The world of architecture is rich and colorful, and there is no way to show them all, such as a small interface, which may reverse our assumptions about understanding.
Our (ArchGuard team) current architecture capabilities are limited - too few senior architects.

So, before implementing such a standardized system of architectural patterns, it is better to try to build a more flexible form: the Architecture Workbench. It can help us better explore the system and is more in line with our initial experience.

What is Architecture Workbench

For the concept of workbench, as an active DSL creator, I am more familiar with Martin Fowler's definition of language workbench in Domain-Specific Languages:

The Language Workbench is an environment designed from the ground up to help people build new DSLs and the high-quality tools they need to use those DSLs effectively.

Therefore, in terms of definition, we refer to the definition of the old horse, and draw on the modern workbench concept, mainly including: RStudio's code documentation, Jupyter's interactive analysis. So, our definition is:

The Architecture Workbench is an environment designed to help people design architectures, evolve architectures, observe architectures, and effectively use the high-quality tools required for architectures, such as interactive architecture development and analysis.

Code documentation is used to help us better manage architecture code and integrate it into the software development life cycle, such as architecture documentation, user stories, continuous integration, etc. The interactive analysis is used to help us effectively reduce the customized code, and then evolve into an API interface and customized code.

For Architecture Workbench, its application includes the following core functions and features:

Design architecture, evolution architecture and observation architecture.
Build a digital twin of the architecture
Descriptive programming and explicit design

Among these features, we believe that building the digital twin of the architecture is the most important part of this workbench, and the basic blocks are: design architecture, evolution architecture and observation architecture.

Basics: Design Architecture, Evolution Architecture and Observation Architecture

When governance architecture, we usually focus on how to design, how to evolve, and observe the state of the architecture at runtime.

Design Architecture. Such a function cannot be understood. The difficulty is, how to provide an efficient design mechanism? Using UI drag and drop, or DSL, there are various games. Generally speaking, it will be more friendly to use the code. In order to visually display the design of the architecture, we will represent it through the architecture diagram, which is the core of the system.

Evolving Architecture. Its core is building an architecture governance model and designing a fitness function of the architecture to guide the system to make orderly changes. From the implementation level, it is to analyze the implemented software architecture (ie code, etc.), design metrics, and provide high-quality tools to guide the architecture back to a reasonable state.

Observation Architecture. That is, to observe the running status of the application of the architecture, it mainly depends on various APM (application performance management, application performance monitoring) tools. In the cloud-native era, related tools are also very popular, such as Skywalking.

A basic architecture workbench should consider the above three elements.

Advanced: Building a Digital Twin of the Architecture

The ideal architecture workbench should provide an interactive mapped version of the architecture. What it means: It's a virtual representation of a system or object throughout its life cycle that we can update based on real-time data and use simulation, machine learning, and inference to help decision-making.

Conceptually, it offers a very good future, the difficulty is how to implement such a system?

We are designing a digital twin version of ArchGuard around these three architectural forms (from an implementation perspective):

Design state. Focus on boundary, concept and interaction.
Development state. Focus on levels, dependencies and modules.
Running state. Focus on performance, link level and resource usage.

This version is equivalent to the design state. When implemented, it will also be restricted by development capabilities and data. In the future, should we also have two-way binding of code and its architectural twin?

Core: Descriptive Programming vs Explicit Design

There are many problems with the various architecture-related specifications that we are familiar with, such as: the specification is not intuitive, the specification is not automated, and so on. For an effective evolutionary architecture, we need to effectively combine specification, code, design, that is, explicit programming and explicit design, which are all concepts from PL (programming language) / DSL (domain specific language).

Illustrative programming (illustrative programming) is used to enhance the interpretability of execution results, such as in the context of architecture governance, by generating architecture diagrams to understand the results of DSL writing. Another type of visual way, such as tables and charts in Excel. For details, see Martin Fowler's related article at the end of the article.
Explicit Design. Explicit is the manual method of accomplishing the desired changes by explicitly writing the instructions to be done. Explicit design is a description language that guides us on how to build a good system, such as a DSL in ADL (Architecture Description Language). In addition to using the language pasted into the architecture description, there is also tool support for supporting the DSL, such as editor intellisense.

These two features are improved around the concept of architecture as code, such as PlantUML, Graphviz and other diagrams as code (Diagrams as code) also have such characteristics.

Other Traits

Of course, in order for this system to work well, we need a series of other features:

Simulation verification of architecture evolution. After building the digital twin of the architecture, we can
Easy to extend functionality. That is, the microkernel architecture is adopted, that is, plug-in, micro-front-end, etc.

There's still a lot to explore, and we'll talk about it after we've built the first version.

How should we design and implement it?

We are now implementing such a system in ArchGuard and it is quite exciting. As an early version it barely worked, here are some ideas for how we built the system.

1. Big data workbench as inspiration

Since the system is a system similar to data analysis, many of our flus come from the field of big data. On the workbench, at first, the idea was to build a compilable report that takes RStudio-like and provides a Jupyter-like interactive experience.

However, due to the poor interactive experience of Jupyter, the downloaded files are not readable and cannot be easily edited + run. Therefore, we use Markdown as the data format, which is convenient for developers to write documents, and can also be opened and edited in any IDE. On the UI, the main editor of ProseMirror is used, that is, the editor of Markdown, and the Monaco Editor is used as the secondary editor, that is, the editor of code.

2. Building REPL-based abstractions

With IDE support in mind, we adopted Kotlin to build the internal DSL, so that no external DSL needs to be built with editor support. Then, Kotlin Scripting Compiler is used as the underlying REPL (read–eval–print loop, also known as interactive top-level component), which is simply the command line mode of the Kotlin language. In terms of form, it is similar to the form of MySQL Workbench, or the CLI mode of Python.

Then, there is the DSL of the abstract system (ArchGuard), the architecture of the design state, and the architecture fitness function.

3. Request and responder operation

With the support of the compiler backend and the Kotlin DSL IDE, we need to build a bridge between the two. Therefore, we use WebSocket as the communication method, and the returned results are processed according to different types. such as:

Integrate the running result of the backend API.
Graphical results. Mermaid, D3.js, Echart.js are for different graphic displays.
Perform the corresponding scan.

It is no different from the usual C/S structure development. Graphical results to support our Descriptive Programming and Explicit Design.

other

We are still designing and building such a workbench, feel free to hook us up: https://github.com/archguard/archguard.

Reference content:

"Domain specific language"
"Evolutionary Architecture"
"Data Analysis with Python"
"IllustrativeProgramming"

ArchGuard: a open-source architecture governance platform

Phodal Huang — Tue, 24 May 2022 13:20:25 +0000

Over the past 10 years, software architecture has undergone tremendous changes, from the earlier popular monolithic MVC architecture to the so-called 5:5 split, meaning from distributed to monolithic. The fact is, there are a large number of software developers who cannot see the whole picture of the system or change from a single-minded approach. As a result, even when using microservices, the implementation is still monolithic, and they become "distributed monoliths."

Architecture governance has become an urgent issue.

The challenges we face

As a software architect or developer, we face many challenges in architecture governance:

Design and implementation mismatch. There is a huge difference between the designed software architecture and the actual implemented architecture. This difference often needs to be coded online or is onlydiscovered later.
No norms/non-compliance with norms. As a senior developer, we have a set of specifications,but many team members aren’t willing to abide by them
Huge amount of code, making it difficult to identify problems. In asystem created by a dozen or more microservices, it’s often difficult to quickly discover the intricate relationships between them.
Errors can occur at every level of the architectural model. Such as API coupling between services, coupling between codes, database coupling, etc.
Architectsand developers themselves lack extensive experience. “I know there is a problem, but I can't tell what is wrong, and I don't know how to fix it.”

As a result, we need a platform / tool to help us resolve these issues. So, using our past experience in software development and refactoring, we (DTO’ consultants) started developing ArchGuard, an architecture governance platform, in 2020/01.

Now, it’s open source: https://github.com/archguard/archguard

What can it do?

ArchGuard performs layered analysis according to the popular C4 architecture model. That is, on the four different architectural views of System Context, Container, Component, and Code, which are different levels of abstraction that correspond to different audiences, such as developers in the team who care about the code’s internal dependencies, and architects who care about dependencies between components and windows.

In its final implementation form, it exists in the form of codebases and documentation. ArchGuard is a code-based static analysis tool, which will also be provided by design in the future.

In ArchGuard, we need to create a series of system components first, that is, to configure the corresponding language and GitHub address so we can just scan the code.

Components / Modules

In the component view, we can see the overall situation of a single project, according to the corresponding code submission history, and unstable code modules:

API declaration and usage:

The architecture is evaluated through the five dimensions of volume, coupling, cohesion, redundancy, and testing. A series of indicators are used to analyze the system’s situation:

System Dependency Analysis: Service Map

Note: This dependency analysis method depends on team developers having a unified coding standard.

For microservices, ArchGuard can automatically analyze the dependencies between different services and visualize the dependencies:

Note: Since ArchGuard used to be microservice architecture, after being merged into a monolith, there can be a situation in which it calls itself.

At the same time, the system can help you automatically analyze which APIs are used and which APIs aren’t (some APIs temporarily cannot be analyzed):

Currently, ArchGuard can support several limited API call identifications such as Spring, RestTemplate, Axios, UMI-Request, etc.

Database Dependency Analysis: Database Maps

Note: This dependency analysis method depends on team developers having a unified coding standard.

For inter-database dependencies, ArchGuard can parse the SQL calls in the code, and try to match the dependencies with different microservices, and then analyze which services are coupled at the database layer. Due to inconsistent coding standards, we could not identify all database tables in the code:

With this dependency, we can see the most frequently used tables in our code. Combined with our functions in code analysis, you can view the call map of the database (in front-end implementation).

Code analysis

For development teams, they can view the dependencies between the project’s modules, packages, classes, and methods on ArchGuard:

Through the above LoginModuleRepository, you can match the results corresponding to the database.

Change Impact Analysis (In development)

One of the features we are implementing is to analyze and configure the system for potential code modification points, and then analyze the impact of changes through dependencies. It can not only help architects analyze the impact of requirements, but also help testers test the content of the system more accurately.

How does ArchGuard handle the functions above?

ArchGuard has two built-in code analysis engines: Bytecode analysis + source code analysis.

Bytecode analysis. As the name implies, it analyzes the dependencies in the code by analyzing the bytecode in the JVM.
Source code analysis. Meaning generating a specific data structure by analyzing the syntax tree of the compiled language.

The source code analysis is mainly static analysis, combined with the Chapi code analysis engine (https://github.com/modernizing/chapi) previously open sourced under the refactoring automation open source organization Modernizing. Chapi is based on the syntax analysis implemented by Antlr and supports mainstream programming languages such as TypeScript/JavaScript, Kotlin, Java, C#, etc. As shown in the table below:

Due to static code analysis, some content isn’t very accurate.

Combined with several ArchGuard Scanner (https://github.com/archguard/scanner) scanning tools to get the data to flow into the database:

scan_git, analyzes basic information such as Git commit history, line count, language, etc.
scan_jacoco, to analyze code test coverage
scan_bytecode, bytecode analysis
scan_sourcecode, source code analysis (including HTTP API analysis, database analysis)
scan_test_badsmell, test code for bad smells
collector_ci, collects history from CI/CD

Most*important thing *

You’re welcome to join the development of ArchGuard, GitHub: https://github.com/archguard/archguard

Componentless: a architecture pattern for low-code age.

Phodal Huang — Tue, 21 Dec 2021 15:04:30 +0000

PS: This is a very fun front-end architecture model that can create unlimited fun. I wrote this article mainly because it is fun, and there is nothing new.

When I create Quake which is a meta-framework for knowledge management, with meta-data and quake component, you can combine any data to any component, like story in calendar with Transflow DSL from('story').to(<quake-calendar>, story is build from meta-data:

I found componentless is the pattern of Quake's low-code design principle, I decide to abstract the patterns of it. I call it Componentless:

Componentless architecture is an architectural pattern, it refers to a large number of rely on third-party components (runtime dependent components rather than compile-time dependent components, that is compilation as a service) or temporary storage of custom code running in the container Frontend application. The three-party components of the application, like the three-party API service, can be independently released and deployed independently, and the application does not need to be recompiled, built, and deployed.

From the name Componentless, you can know that its target is the serverless-like back-end architecture pattern. Therefore, the definition is quite similar to that of the Serverless architecture. That's why, we define it as a componentless architecture. You don't need to write any components, you only need to write logic code or DSL to achieve their combination. Moreover, we can only provide a DSL + universal URL, and the browser will complete the automatic construction and operation of the application according to the DSL.

Quake online demo: https://quake-demo.inherd.org/, try typing from('stroy').to(<quake-calendar>), the <quake-calendar> can be any quake-component (like <quake-component>, we only have two components in 2021.12.21) or any ionic component.

Quake source code: https://github.com/phodal/quake/

Evolution of front-end and back-end architecture

Previously, it was quite interesting to often communicate with others about the application of domain-driven design (DDD) in the front-end. As a “nine and three-quarters”/10 DDD bricks, in the past, I always felt that domain-driven design is not suitable for the front-end. Clean front-end architecture is what people need, but design + getting started is slightly more difficult.

In this year, I have used DDD design and planning for multiple back-end applications, and I have a new experience (although it still doesn't work). The front-end can have a DDD-like approach, but the approach is completely different from the back-end. The back-end uses model and function as the basis of two different programming styles, and the front-end uses components + events as the basis of the programming style. Components are destructible, and events are orchestrated by designing event streams.

Therefore, you don't directly apply the idea of back-end DDD to front-end applications, unless the logic of your application is focus on the front-end.

Microservices and micro frontends

For most of today's systems, they still remain in a state of "back-end microservices, front-end "big mudball"." The back-end microservices have been disassembled into individual microservices according to "Conway's Law" (of course, unreasonably splitting the microservices is another problem), while the front-end is still in a state of a big mud ball. Therefore, the micro-frontend is used as one of the (not the only) technologies to solve the organizational structure alignment and implement the architectural model of rapid release and online. It can split a single large application into multiple smaller autonomous applications, but they are still aggregated into one. It can be used to solve the migration of legacy systems, unify user experience, help multi-team collaboration, etc.

When migrating back-end systems, we use DDD (Domain Driven Design) to find a reasonable basis for the design of microservice architecture. Microservices have become a way for us to transform the legacy system. We start with one module and one function, and gradually replace the old single application until the entire system is replaced. This replacement mode is quite similar for front-end applications.

Therefore, after the transformation of the micro front end, the structure is aligned, and the personnel is aligned. Everyone is happy.

Going forward, how should we continue to evolve the system?

Serverless and ComponentLess

In 2017, after learning about DDD and Serverless, and create the "Serverless Application Development Guide" (https://serverless.ink/), I have been thinking about how to apply Serverless-like ideas in the front-end? So, there was an idea about the cross-frame component library: "The second half of the front-end: building a cross-frame UI library", but these domestic companies that write component libraries do not have such a bold idea, it is a pity-only the version number + 1, what others do follow? There is also an interesting story line. After experiencing the enthusiasm of low-code programming, I rethinked the future of front-end and back-end: "Front-end and back-end integration: Will the front-end and back-end separation die?".

At first, I thought no-code programming was a ComponentLess direction, but a research found that it was not. Nocode programming tends to visualize programming, while Componentless tends to use DSL programming. At this point, I prefer to use Web Components + WASM technology to build a new front-end architecture.

Until I recently reapplied this idea in the open source knowledge management tool Quake, I found it particularly interesting, so I wanted to write an article to introduce the related idea-after all , The market has accepted the Serverless concept and the micro front end concept. Then, the remaining questions become very simple.

Componentless architecture

Continue back to the definition at the beginning:

Componentless architecture is an architectural pattern, it refers to a large number of rely on third-party components (runtime dependent components rather than compile-time dependent components, that is, compilation as a service) or temporary storage of custom code running in the container Front-end application. The three-party components of the application, like the three-party API service, can be independently released and deployed independently, and the application does not need to be recompiled, built, and deployed.

Simply, what a componentless thing needs to do is to turn the component into a runtime service instead of a compile-time dependency in the past. When all the components become a kind of infrastructure, we no longer need these components, and then let the components disappear from the application development side, and achieve a state where the application does not require components. In this way, it has also become a LowCode type system, with simple code generation, it can reach the state of NoCode.

From a formal point of view, the use of micro-front-end related technologies can provide a series of basic technologies required by a componentless architecture. Among them, the simplest way is to use: Web Components. So, let us first look at an example of a Componentless architecture based on Web Components.

Example: How to move towards a Componentless architecture?

In terms of the process, it can be divided into three steps:

Decompose the application using Web Component
Split more components to eliminate components
Build a generative low-code model

The remaining part is fill-in-the-blank programming.

1. Use Web Component to decompose the application

Let's look at an example first. For example, our front-end part has two micro-applications, A and B. The granularity is already very small, but it is still an application-level application. Application B is built using Web Components technology, and two tripartite Web Components components are introduced into micro-application B. In a conventional front-end application, if we update these two components, the corresponding application needs to be recompiled and released again.

For now, with the support of Custom Element + Shadow DOM, we only need to update the link to the script tag of the component library, or the cache.

2. Split more components to eliminate components

Next, let us further optimize, remove all the internal components of application A and application B, externally build these components into a set of components according to functional modules. These component sets can be divided by functional teams.

These are not important. Now there are very few "components" in our application-we still have some components for orchestrating these components + some additional business logic.

3. Build a generative low-code model

Now, let’s review the "hello, world" function written in Serverless (AWS Lambda, they don’t pay for advertising):

module.exports.hello = (event, context, callback) => {
   callback(null, "hello, world");
};

When using a framework like Serverless Framework, we only need to fill in our business logic on this function, that is, fill-in-the-blank programming. For the front end, the process is similar. We have data and our target components. Only a limited code generation function is needed. That is, we only need to generate an empty function to be improved, such as Transflow in Quake: from('todo','blog').to(<quake-calendar>), the generated function and logic (part of the code example ):

const tl_temp_1 = async (context, commands) => {
const el = document.createElement('quake-calendar');
    ...
return el;
}

At this time, you only need to ensure that the routing and functions are not modified, and the remaining part is to fill in the blanks for data processing.

Migration to Componentless

In addition to the above-mentioned direct decomposition method, there are other gradual migration methods.

Migration method 2: new embedded in old

Use new technologies and frameworks to create application shelves.
Extract the Web Component and insert it into the old component, and then change the public capabilities.
Embed old wheels in new applications.

Migration method 3: old embedded in new

Build a new Web Component component. Cooperate with monorepo management
Embed components into existing applications.
Improve the componentless architecture mechanism.
Build a low-code orchestration model.

Componentless architecture concept

From the current personal understanding, its core concept is: Components are "services." That is, components can be deployed and updated freely, just like services. After the component is updated, the application has also reached the update of the application in a sense.

In addition, there are such as:

Automated environment isolation. Christmas is coming soon
Generate low code. The real front end glue

Componentless architecture issues

In addition to the many advantages mentioned above, it also has a series of shortcomings to be solved:

Browser compatibility. Web Component2 compatibility issues
Test difficulty. Free architecture often means the cost of testing, which is similar to microservices and serverless at this point. More end-to-end testing will be required to ensure the quality of the project.
The division basis of component modularization. When we build a set of components, we need to find a way to plan rationally.
Monorepo management. The more repo, the more complicated the management. Need to introduce tools such as nx and pnpm for management.
Upgrade strategy. That is, the upgrade strategy of the application and the component set should remain inconsistent. ...

Advantage scenario: combined with lowcode

In a sense, componentless architecture is a generalized low-code implementation mode. Because of the more independent component model, the low-code system it builds is more interesting:

Configuration is runtime. It is similar to the process-oriented style of Oracle DB, and realizes new features quickly on the line.
Fill-in-the-blank programming for code generation. As mentioned in the above example, basic function codes can be generated, and then developers can add code logic.
Low code based on stream orchestration. The same applies to the traditional low-code architecture model.
DSL style low code. Such as Quake based on DSL to build.

It's just that, in terms of mode, there is not much difference.

Componentless patterns

None of the above is interesting. After we adopt Web Component as the implementation technology of componentless architecture, there will be more room for architectural display. Web Components is already a very good container similar to Docker, which can play various fancy containerization modes. We tried some patterns at Quake, which brought a series of challenges, but it was also very interesting.

Adapter: Compatible with existing components.

Based on the features of WC, encapsulating the components of the existing mainstream frameworks such as Angular, React, Vue, etc., can quickly provide such capabilities. For example, the QuakeTimeline and QuakeCalendar we provide in Quake are all in this way. React components are packaged as Web Components:

class ReactElement extends HTMLElement {
...
}
customElements.define('quake-calendar', ReactElement);

Since the WC components are exposed to the outside, it doesn't matter what front-end framework is used.

Ambassador pattern

In the cloud-native model, the Ambassador model can create services or applications on behalf of consumers and send help services for network requests. The same event can also be encapsulated by components,

const fetchEl = document.createElement('fetch-api');
fetchEl.setAttribute("url", "/action/suggest);
fetchEl.addEventListener("fetchSuccess", (res: any) => {
let response = res.detail;
loading.onDidDismiss().then(() => {});
callback(response);
})

However, I wrote this just for fun. I created a Loading component and inserted the <fetch-api> component in Loading to initiate an HTTP request. After the request was successful, the DOM was destroyed.

In this way, I only need to replace this request component to replace all request APIs.

Infinite nesting "Dolls" patten

In the normal pattenrn, we call the B component in the A component, then in theory, we don't need to call the A component in the B component, it will form a circular reference, but it becomes a function in Web Components.

For example, in Quake's markdown rendering engine <quake-render>, the <embed-link> embedded in the page is rendered conditionally, and the embedded page of the is also markdown, so we need a , So you can "mock doll" infinitely, until the current page of the browser is hung up.

In terms of usage, the two components A and B do not have such a mutual calling relationship.

PS: Actually this is a bug. Later I thought it was a features.

Sidecar pattern

In the cloud-native patterns, the sidecar model refers to the deployment of application components into separate processes or containers to provide isolation and encapsulation. In this regard, it is also very simple for Web Components.

Other parterns

There are still many, you can play slowly when you have time.

Summarize

Think outside the frame and think about the problem, and you will find all kinds of very interesting things.

Documentation as Code practises

Phodal Huang — Mon, 20 Apr 2020 09:13:45 +0000

Documentation as Code. Documents are written in a domain-specific language like code, and are managed with reference to software development methods (such as source code management and deployment). It can be edited, previewed, viewed with the help of specific tools, or deployed to the server through a dedicated system. A common architectural pattern of document coding for non-technical personnel is: edit-release-development separation "

In the past month, I am developing a new document system based on Git + Markdown + DSL which is call Ledge. I customized a markdown-based markup language to support the file system of radar charts, bar charts, mind maps and other charts.

Also in the past few years, we have been discussing a variety of coding, infrastructure coding, design coding, requirements coding ... In my article "Cloud development: R & D as Code", I designed a fully coded software development process. And today we will discuss another interesting existence: document as code.

Introduction 1: Architectural Decision Record: Formatting Documents

Three years ago, when I first came into contact with the concept of "architectural decision record", I was attracted by its concept:

Use lightweight text formatting language to describe major decisions Version along with the code
Use a specific document format (title, context, decision, status, consequence)

Later, I wrote an ADR tool using Node.js + TypeScript. Now, in most of my open source recommendations, I will use it to manage some technical decisions. Because this document system designed based on this theory is really great, I can query:

When a technical decision occurs and the architecture changes, the corresponding modifier
Backtrack all technical decisions and sort out the architecture development process
All decisions are recorded in the version control system and can be recovered
Easy to manage and maintain

For a long-term development system, it is really very useful.

Introduction 2: Static Site Generation: Data as Code

Static site generation is a hybrid Web development method that deploys by deploying pre-built static files to allow developers to build server-based websites locally.

When GitHub Pages became the programmer's preferred blog / content / document server, he / she also adopted the technique of static site generation. Static site generation has various advantages:

Better reliability, safety, stability, usability, etc.
Version control
Easy to test
Easy to practice continuous deployment. Submit to go online
Flexible and easy to customize

In fact, the most important thing that static site generation does is to code the data in the database. When using a CMS such as Wordpress, we store the data in the database to achieve CRUD for the data. An article becomes a fragment in the database binary file.

Subsequently, the second thing the static site generation tool did was to visualize the text content for people to read. In this way, we have achieved release-development separation.

Introduction 3: Customized markup language: expansion

When codify data, we faced a very big challenge: easy-to-write and readable markup languages (such as markdown) only designed the form of the content, and lacked other information related to the content, such as creation time, author, modification time and many more.

So each static site generator customized its own markdown and added some additional information, such as hexo adopts the form of: year-:month-:day-:title.md to manage the date and title of the article. In this way, there is no need to read the Git information of this article to construct the entire information.

The same is true of GitHub Flavored Markdown, which we are familiar with, extending the markdown data field through a compatibility mode that does not significantly disrupt the content format.

In addition, we can customize charts and mind maps based on markdown data.

Introduction 4: Edit-Release-Development Separation: For Non-Technical Personnel

Designing for non-technical personnel is a major challenge for code documentation. As a programmer, we think the markdown syntax can't be simpler, but it's not the case for non-technical people. He / she needs: an easy-to-use visual programmer. To achieve such a goal, we need to make some changes in the architecture, we can try to use the "edit-release-development separation" model to solve this problem.

That is, we split the process into three steps:

Editors can use commonly used editors or customized editors
Developers, writing content display
When publishing, integrate the two parts of the code

We can still choose to manage content with source code management. Just change the database interface to the Git server interface-of course they are slightly different. However, converting local Git to Git remote is basically the same.

As a result, in the end our cost will be to transform a markdown editor based on Git.

Document as code

Perfect, I once again expressed the central idea in the introduction.

Why do you need to document as code?

The main reasons are: if the documentation is not coded, there is no possibility of refactoring.

The remaining reasons are:

Binary documents are difficult to manage version. Imagine 2020-02-30.docx and 2020-02-31.docx.
Can't accurately know who is the editor of the document, everyone may be admin, or Zhang San at the meeting
Can't find which is the latest document
The documentation is terrible, but you ca n’t reconstruct the binary
Vendor binding
...

There should be more.

What is Documentation as Code?

Back to the topic:

Documentation as Code. Documents are written in a domain-specific language like code, and are managed with reference to software development methods (such as source code management and deployment). It can be edited, previewed, viewed with the help of specific tools, or deployed to the server through a dedicated system.

It has such characteristics:

Use markup language to write content. Such as markdown
Version control can be performed through the version control system. Like git
Programming experience consistent with programming (except that the content cannot be written for testing)

And an efficient document coding system also has such characteristics:

Continuous deployment, that is, content can be automatically published after modification.
Organize content indexes with specific forms. For example, to organize content in the form of knowledge base.
Specific text format. Such as architecture decision records, static content generation, for use to provide a better user experience
Supports REST API. To modify the content through the editor
Can support multiple ways of output. Such as the standard HTML of the website, or Docx, Latex, etc.
Support editing and proofreading workflow
Support search
Multiplayer collaboration

In fact, most of the teams do not need the advanced features mentioned above, and they already have mature solutions.

How to design a documentation as Code system?

In fact, we have marked the elements we need in four primers:

Use formatted documents
Release the system with the help of static site generation technology
Expand capabilities through custom markup languages
Implementation of the editor for non-technical personnel

Design a markup language and its extended syntax, and then implement it.

1. Establish key factors

Considering that my colleagues and I have recently implemented such a system, I still endure the pain of my hands and briefly talk about how to make such a system. The main factors we considered are:

Chart rendering
Flowchart rendering
Visual display

Because the chart converted from DSL is easy to modify and can be indexed. Since then, we:

Expand these capabilities through Markdown's Code syntax
Use markdown tables and lists to provide data
Use D3.js to support flowchart drawing
Use Echarts for charting
Try to use SVG images
...

2. Implement an MVP

We used Angular + GitHub to quickly implement an MVP:

We use Git as the database. It can achieve the purpose of multi-person collaboration, and can track all changes
We use GitHub Pages as the server. As soon as the document or code is modified, the latest document is deployed.
We use marked.js, which allows us to quickly expand the syntax.
Using textarea combined with markdown to make a simple editor.

Subsequently, we carried out a quick iteration on this basis.

3. Extended syntax

We used the markdown code as the DSL of the chart, extending some of the syntax:

echarts. Render Echarts charts directly
mindmap. Markdown List becomes mind map
radar. Markdown List to radar chart
process-table. Chart with process
process-step. Another chart with process
pyramid. Pyramid graphics
quadrant. Four-quadrant graph
class. Directly call CSS class
graphviz. Use Dot to render pictures
mermaid. Use mermaid visualization
webcomponents. Call the WebComponents component
toolset. Call Toolset related components
- slider. Trade-off slider
- line-chart. Chart Therefore, for users, only need to write the following code:

\`\`\`radar
  -Quality maturity evaluation model
   -Built-in quality: 3-> 4
   -Optimize business value: 2-> 2
   -Uniform quality, visualization: 1-> 5
   -Full participation: 3-> 4
   -Fast delivery: 4-> 5
   -Test as an asset: 2-> 3
   -Quick feedback: 5-> 5

config: {"legend": ["Current", "Future"]}
    \`\`\`

You can generate the corresponding chart:

We also input JSON through config and do some lazy processing (don't exhaust yourself).

3.1 Rewrite the markdown renderer

The biggest challenge we encountered in this process was that as we continued to expand the markdown syntax, the relevant code has become a big ball of mud. Therefore, we have to rewrite this part of the code:

Resolve token with the help of marked.js lexer
Generate new token based on token modification
Traverse the newly generated token and render the element
Combine virtual scrolling to solve performance problems

It has been open sourced on GitHub and the corresponding npm package has been released: @ledge-framework/render.

4. Publish this project

We have published this documented system on GitHub, and you can participate in its use and development.

GitHub: https://github.com/phodal/ledge

Project homepage: https://devops.phodal.com/

Summary

Then, you become a Markdown engineer, D3.js designer, Echart configuration administrator.

Clean Architecture for frontend in Action

Phodal Huang — Sun, 13 Oct 2019 12:26:03 +0000

中文版本（Chinese Version）：《Clean Architecture 实施指南》

In the previous article Clean Frontend Architecture, we introduced how to use Clean Architecture in the frontend. In the past few months, we have implemented the Clean Architecture architecture and have proven that Clean Architecture can also work very well on the frontend.

Clean Architecture + MVP + Componented-based Architecture

Before we get started, let's take a look at the final architecture of Angular apps that use Clean Architecture:

In the figure, we split the architecture for this part:

Data layer. That is, the processing layer of all frontend and backend interactions, from the request to the return result, only returns the value and field which required by the front end.
MVP layer. MVP is not the focus of this article, but interestingly, in Angular applications, the module layer can correspond to the backend service. The page under the module layer can also be split according to this.
The component layer in MVP. A reasonable planning of the component layer will make our componet layer clean, not just the domain layer. Components, the hard part is to meet the scene, not split, combine and package.

Here, we have less part of the style layer. As a result, the use of various CSS preprocessors to organize code is very mature; second, CSS has not hurt so much in today's infrastructure.

We have added more implementation details than our old architecture diagram:

However, the former is qualified and the latter is the architecture under general conditions.

Context of implementation

Clean Architecture is not a silver bullet, it is suitable for us, it does not mean it is suitable for you. Especially if you are used to free style and autonomous project development, then the strong standardization of Clean Architecture + Angular is not necessarily for you. However, at the same time, if your team is large and there are more junior developers, then I think normalization can help you reduce problems - easy to maintain.

So, let me introduce some contexts that are good for us:

Implementation of DDD's microservice backend architecture.
A member who is interested in implementing a full-stack team.

There are other Angular frameworks that are more suitable for the enterprise (pursuing standardization), such as the junior developers, so the specification is more, but better maintained. However, the remaining factors, for our architecture: helpful, but not too big.

Implementing DDD's microservice backend architecture

DDD is just a set of software development methods, and different people understand the DDD self-differentiation. In a specific scenario, when DDD is used for microservice splitting, each subdomain is a specific microservice. Under this model, we need to have a naming pattern to distinguish each service, and one of them is a URL. Each service has its own unique URL prefix/path, and when these services are exposed, the corresponding front-end domain layer can be produced—even if it has not participated in an event storm or domain division.

For example: /api/payment/, /api/manage/ can clearly split the front domain data layer.

At the same time, if the backend then names the Controller according to the resource path, such as /api/blog/blog/:id,/api/blog/blog/category/:id, then the front end can Clearly divide them into the same repository. Of course, once the backend design has problems, the front end can be clearly noticed.

Full-stack team

In the past, we built our team as full-featured team in a ThoughtWorks, each team members excel in a particular area, but in other areas, such as good at the frontend and can do backend. It can reduce communication costs to some extent. And that means we have a lot of cost for knowledge transfer. Therefore, we used pair programming to let new people talk about project-related sessions.

So, when you decide to become a full-featured team (front and back + business analysis), then you will encounter such problems:

Find the front-end code of the app, how to find it quickly?
Find the corresponding backend code,
Front and back model corresponding
......

Keeping the front and back ends as consistent as possible becomes a new challenge.

Directory as layered architecture

In a sense, Clean Architectute is an implementation of normalization and templating. At the same time, its three-layer mechanism in the data layer makes it exist in two layers of anti-corrosion layer, usecase can be used as a buffer layer for services, and the repository layer can isolate back-end services and models.

In many architectural designs, layered architecture is the easiest to implement because directories are layered. The catalog is a specification that you can see at a glance. Once misplaced, you can see it at a glance.

MVP Stratification (Directory Partitioning)

From the catalog results, our division is not much different from the general Angular application.

├── core           // core code, including basic services and basic code
├── domain         // domain layer code, containing separate Clean schema content for each domain
│ └── elephant     // a specific domain
├── features       // public domain components
├── presentation   // domain logic page
├── pages         // public page
└── shared        // shared directory

We will:

The business pages are placed in the presentation directory.
Public pages (such as 404) are placed in the pages directory.
The business components shared by the business page are placed in the features directory.
The remaining common parts are placed in the shared directory, such as pipes, utils, services, components, modules.

Sample code can be found at: https://github.com/phodal/clean-frontend

domain + data Layer: Vertical + Horizontal Layered

The domain layer in the above directory, the example structure is as follows:

├──model
│ ├── elephant.entity.ts       // Data entity, a simple data model used to represent the core business logic
│ └── elephant.model.ts        // core business model
├── repository
│ ├── elephant.mapper.ts       // Mapping layer for core entity layer mapping, or mapping to core entity layer. Model conversion
│ └── elephant.repository.ts   // Repository for reading and storing data.
└── usecases
    └── get-elephant-by-id-usecase.usecase.ts // Use cases, built on top of core entities, and implement the entire business logic of the application.

The relevant explanation is as above, here is not Ctrl + V / Ctrl + C.

It is worth noting that we are adopting a vertical + horizontal double layered model, and vertical response to domain services. It is suitable for microservices architectures without BFF, especially for microservices backend applications that use DDD.

Mapping Domain Services

In the previous section, the front-end domain + layer layer actually mapped the backend services. When the current end initiates a request, its flow is generally like this:

Component / Controller -> Usecase -> Repository -> Controller (backend)

The corresponding return order is:

Controller (backend) -> Repository -> Usecase -> Component / Controller (front end)

To do this, we correspond the Repository to the Controller on the back end. And because of the simplification of the service, most of our usecase also corresponds to the naming in the repository.

repository naming: URL naming

To name it without looking at the backend code, we use URLs to name methods in the repository and repository. If there is one

URL	Explanation	Abstract
`/api/blog/blog/:id`	/API/Micro Service Name/Resource Name/Resource ID	HTTP Verb + Resource + Behavior

Ever since, the corresponding repository should be named blog.repository.ts. The name of the corresponding repository is also get-blog-by-id. Similarly, the repository corresponding to the URL /api/blog/blog/:id/comment is get-comment-by-blog-id.

Well, yes, it is consistent with the access to the database.

usecase naming

Since we don't involve complex APIs, the common behavior is as follows:

General verb: get / create / update / delete / patch
Unconventional: search, submit

Haha, is it similar to the repository.

Clean Architecture's MVP layer practice

In fact, the MVP layer here, the main content is the component architecture. This part of the content has been in the previous article ( Component-based Architecture's patterns ) introduced, not detailed here. A brief introduction is:

Basic component library, such as Material Design
Encapsulation package components. Additional three-party component libraries, such as Infinite Scroller, must be used after encapsulation.
Customize basic components.
Domain specific components.

The above four parts build a common component library module for the entire system. This is followed by domain-related components and page-level components.

Domain related components. Share logical components between different modules and pages.
Page level components. Share pages between different modules and routes.

Well, there are so many things in this part.

Clean Architecture Domain + Data Layer Practice

Well, the other parts are not much different from normal project development. So we can focus on the Domain + Data layer.

DDD ApplicationService vs Multiple Usecases

In DDD practice, it is natural to adopt a top-down implementation. The implementation of ApplicationService follows a very simple principle, that is, a business use case corresponds to a business method on the ApplicationService. ¹

A little different is that the Clean Architecture recommended by us is: A business use case (usecase) corresponds to a business class. That is, in the same business scenario, the frontend is a bunch of usecase files, and the back end is an applicationService. So, in this scenario, the frontend has them:

change-production-count.usecase.ts
delete-product.usecase.ts

The backend is OrderApplicationService.java, which has multiple methods.

usecases + repository vs services

If our usecases + repository does the same thing as a service, isn't it good to just use serivce? Using only service has this problem:

repeated API calls
Calling context not clearly defined

So, use usecase:

More templated code
More stratification

The reuse rate of Usecases is extremely low, and the project will dramatically increase the class and duplicate code. Therefore, we try to increase the maintainability of the architecture with more code. PS: More code, it may also reduce the maintainability of the code, but in today's intelligent IDE, this should not be a problem.

Usecases as a logical layer / anti-corrosion layer

However, Usecases brings a layer of anti-corrosion while bringing more code. It is responsible for the following responsibilities:

Business logic processing. Process the necessary content before the data is passed to the back end.
Return to data management. From the data returned from the backend, build the results needed for the front end. This can be done in usecase when multiple APIs need to be called.
Input parameter management.

Therefore, the Usecases layer is very useful when the current end is given too much business logic. Conversely, if the logic is placed in the BFF layer, the Usecases layer becomes a bit sloppy. But it is still a very good anti-corrosion layer.

Model Management

As we work with Usecase, we need to solve the frontend model problem. The backend has multiple microservices and multiple projects, each with its own model. And if there is only one project in the frontend, the frontend model management becomes a pain point. Because in different contexts, the backend model is different. That is to say, in different APIs, the models are different, and these models are customized according to the business, and finally aggregated together at the front end.

At this time, it is possible to have multiple different models for the same resource. So either:

The frontend has a one-to-one model. It is more troublesome to manage.
Use the same model. You cannot use type checking to reduce bugs.

Currently, I can't figure out a better solution. We use the second way to manage model, after all, it is easier to manage.

Here are a few of our types of classification and management:

Request Model / Response Model. That is, the request parameters and the return model (modified, for frontend view) are placed in the corresponding .model.ts directory of the service.
Response Entity. When using the backend to return results directly, the name has entity, otherwise model is used.
View Model / Component Model. When applied to the encapsulation of business components, incoming parameters are passed in by model.

Do you have better practices?

Question

Is it clean? not yet

Framework dependent form validation

Since the Angular framework itself provides powerful Reactive Form functionality, we use Reactive Forms in most of the form design, rather than through Entity. This allows us to interact in this part of the UI, relying on the Angular framework, rather than implementing it ourselves.

If you use an Entity mode such as DDD, or use a validator. Later, we also need to develop our own form validation mode, similar to this:

{
    Validator: RegExp,
errorMessage: string
}

And it means a lot of development costs. However, fortunately, we can try to generalize it.

Next step

Clean form. As mentioned above.
Code generation. Although we have already used Angular Schematics in our project to generate template code. But I believe that in the next step we can use tools to generate pages.
Architectural Guardian. With a hierarchical structure, it is easier to determine the hierarchical relationship.
Try Other frameworks.

https://insights.thoughtworks.cn/backend-development-ddd/ ↩

Micro-frontend Architecture in Action with six ways.

Phodal Huang — Wed, 26 Jun 2019 05:11:32 +0000

Micro-frontends is a microservice-like architecture that applies the concept of microservices to the browser side. Transforming to a mono-like applications from a single, single application to an application that combines multiple small front-end applications. Each frontend application can also be standalone run, independent development, standalone deployment.

At the same time, they can also be developed in parallel with sharing components - these components can be managed via NPM or Git Tag, Git Submodule etc.

Note: The front-end application here refers to a single-page application separated from the front and back. It is meaningful to talk about the micro-frontends on this basis.

TOC:

Thinking in Microfrontend
Why micro-frontends be popular – aggregation of web applications
- Migration frontend legacy system
- Backend decoupling, frontend aggregation
- Compatible with legacy systems
Six ways to implement a micro-frontends architecture
- Basic: Application Distribution Routing -> Route Distribution Application
  - Backend: Function Call -> Remote Call
  - Front End: Component Call -> Application Call
- Route-dispatch micro-frontends
- Create a container with iFrame
- Homemade micro-frontends framework
- Combined integration: Widging applications
- Pure Web Components technology build
- Build with Web Components
  - Integrating existing frameworks in Web Components
  - Web Components integrated into existing frameworks
- Compound type micro-frontends
- Microfrontend Quick Selection Guide
- Comparison of micro front-end solutions: a brief comparison
- Comparison of micro front-end solutions: complex ways
How to deconstruct mono front-end application - microservice splitting of front-end applications
- front end microservice
  - Independent development
  - Independent deployment
  - Do we really need technology independently?
  - Does not affect the user experience
- Micro-frontends design concept
  - Design Concept 1: Centralized Routing
  - Design Concept 2: Identification Application
  - Design Concept 3: Life Cycle
  - Design Concept 4: Independent Deployment and Configuration Automation
- Practical micro-frontends architecture design
  - Independent Deployment and Configuration Automation
  - Inter-Application Routing - Events
Four to split large Angular application to micro-frontends
- 1, front-end micro-services: routing lazy loading and its variants
- 2, microservice solution: sub-application mode
- Solution comparison
  - Standard LazyLoad
  - LazyLoad Variant 1: Build-time integration
  - LazyLoad variant 2: Post-build integration
  - Front-end micro-service
- Total contrast

Why micro-frontends be popular – aggregation of web applications

Adopting new technology, more is not because of advanced, but because it can solve the pain.

In the past, I have always had a doubt as to whether people really need microservices and whether they really need micro-frontends. After all, there is no silver bullet. When people consider whether to adopt a new architecture, in addition to considering its benefits, it still considers the large number of risks and technical challenges.

Migration frontend legacy system

For past years, I have received some consultations on how to implement micro-frontends architecture. In the meantime, I found a very interesting thing: Resolving the legacy system is the most important reason people use the micro-frontends solution.

In these consultations, the situation encountered by the developers is similar to the situation I encountered before. My scenario is: design a new front-end architecture. They began to consider frontend with micro-services because of the legacy system.

In the past, single-page applications written using Backbone.js, Angular.js, Vue.js 1 and other frameworks have been running steadily online and have no new features. For such applications, there is no reason to waste time and paty effort to rewriting old applications. Applications in here, using old, no longer used technology stacks can be called legacy systems. However, these applications need to be combined into new applications. The more I've encountered is that the old app was written in Angular.js, and the new app started with Angular 2+. This is a very common technology stack for a business-stable IT team.

Under the premise of not rewriting the original system, it is possible to extract manpower to develop new business. It is not only a very attractive feature for business people; it is also quite challenging for technicians not to rewrite old business and to make some technical challenges.

Backend decoupling, frontend aggregation

A selling point for front-end microservices is also here, to be compatible with different types of front-end frameworks. This reminds me of the benefits of microservices and the reasons why many projects fall into microservices:

In the early, a big selling point for back-end microservices was that different technology stacks could be used to develop background applications. However, in fact, organizations and institutions that adopt a micro-service architecture are generally medium to large-scale. Compared with small and medium-sized, the selection of frameworks and languages is more stringent, such as limiting the framework internally and limiting the language. Therefore, it is almost rare to fully utilize the different technology stacks to take advantage of microservices. In these large organizations, the main reason for adopting microservices is to use the microservices architecture to decouple inter-service dependencies.

In the frontend micro-services, it is exactly the opposite. The result people want more is aggregation, especially those applications of To B(to Bussiness).

In the past two or three years, mobile applications have shown a trend, users do not want to install so many applications. And often a large commercial company offers a range of applications. These applications also reflect, to some extent, the organizational structure of the company. However, in the eyes of users they are a company, they should only have one product. Similarly, this trend is also emerging on the desktop web. Aggregation has become a technology trend, and the aggregation in the front end is the micro-service architecture.

Compatible with legacy systems

Then, at this time, we need to use new technologies and new architectures to accommodate and be compatible with these old applications. The front-end micro-services just fits the selling point that people want.

Six ways to implement a micro-frontends architecture

Combined with my practice and research in the micro-frontends in the past six months, the micro-frontends architecture can generally be implemented in the following ways:

Use HTTP server routing to redirect multiple apps
Design communication and loading mechanisms on different frameworks, such as Mooa and Single-SPA
Build a single application by combining multiple independent applications and components
iFrame. Use iFrame and custom messaging mechanisms
Build an app with pure Web Components
Building with Web Components
Seperate application in build-time (TBC)
Seperate appliation in local build time (TBC)

Basic: Application Distribution Routing -> Route Distribution Application

In a monoli front-end, mono back-end application, there is a typical feature that routing is distributed by the framework, which assigns routes to corresponding components or internal services. What the microservice does in the process is to call the function call into a remote call, such as a remote HTTP call. The micro-frontends is similar, it is to change the component call** in the application into a more fine-grained inter-application component call, that is, we just distribute the route to the application component execution. Now you need to find the corresponding application based on the route, and then distribute it to the corresponding component by the application.

Backend: Function Call -> Remote Call

In most CRUD-type web applications, there are some very similar patterns, namely: Home -> List -> Details:

Home page for displaying specific data or pages to users. These data are usually a finite number and are of multiple models.
List, the aggregation of the data model, which is typically a collection of data of a certain type, can see as many ** data summaries** (such as Google only returns 100 pages), typically see Google, Taobao / Ebay, Amazon search results page.
Details, showing as much content as possible for a single piece of data.

Here's an example of a Spring framework for returning to the home page:



@RequestMapping(value="/")
Public ModelAndView homePage(){
   Return new ModelAndView("/WEB-INF/jsp/index.jsp");
}

For a detail page, it might look like this:



@RequestMapping(value="/detail/{detailId}")
Public ModelAndView detail(HttpServletRequest request, ModelMap model){
   ....
   Return new ModelAndView("/WEB-INF/jsp/detail.jsp", "detail", detail);
}

So, in the case of microservices, it will look like this:



@RequestMapping("/name")
Public String name(){
    String name = restTemplate.getForObject("http://account/name", String.class);
    Return Name" + name;
}

In the process, the backend has a service discovery service to manage the relationship between different microservices.

Front End: Component Call -> Application Call

Formally speaking, the routing of the single front-end framework and the single-end back-end application are not much different: Return the templates of different pages according to different routes. An Angular examples:



const appRoutes: Routes = [
  { path: 'index', component: IndexComponent },
  { path: 'detail/:id', component: DetailComponent },
];

And when we micro-service it, it may become the route of application A:



const appRoutes: Routes = [
  { path: 'index', component: IndexComponent },
];

Plus the route of application B:



const appRoutes: Routes = [
  { path: 'detail/:id', component: DetailComponent },
];

The key to the problem is: How to dispatch routes to these different applications. At the same time, it is also responsible for managing different front-end applications.

Route-dispatch micro-frontends

route-distributed micro-frontends, which distributes different services** to different, independent front-end applications by routing. It can usually be implemented by a reverse proxy of the HTTP server, or by the routing that comes with the application framework.

For the moment, the micro-frontends architecture through route distribution should be the most popular and easy-to-use "micro-frontends" solution. But this approach looks more like an aggregation of multiple front-end applications, that is, we just put together these different front-end applications to make them look like a complete whole. But they are not, every time a user applies from A to B, they often need to refresh the page.

In a project a few years ago, we were working on a legacy system rewrite. We have a migration plan:

First, use static website generation to dynamically generate the home page
Second, refactor the details page using the React stack
Finally, replace the search results page

The whole system is not a one-time migration, but a step by step. So when we need to complete the different steps, we need to go online, so we need to use Nginx for route distribution.

The following is an example of a Nginx configuration based on route distribution:



http {
  server {
    listen       80;
    server_name  www.phodal.com;
    location /api/ {
      proxy_pass http://http://172.31.25.15:8000/api;
    }
    location /web/admin {
      proxy_pass http://172.31.25.29/web/admin;
    }
    location /web/notifications {
      proxy_pass http://172.31.25.27/web/notifications;
    }
    location / {
      proxy_pass /;
    }
  }
}}
}

In this example, requests for different pages are distributed to different servers.

Later, we used a similar approach on other projects, the main reason is: cross-team collaboration. When the team reaches a certain size, we have to face this problem. In addition, there is the problem of Angluar cliff-style upgrade. So, in this case, the user foreground uses Angular rewrite, and the background continues to use Angular.js and so on to keep the technology stack. In different scenarios, there are some similar technical decisions.

So in this case it works for the following scenarios:

The difference between different technology stacks is relatively large, and it is difficult to be compatible, migrated, and modified.
The project does not want to spend a lot of time on the transformation of this system
Existing systems will be replaced in the future
System functions are perfect, there are no new requirements

In the case of satisfying the above scenario, if it is for a better user experience, it can also be solved by using an iframe.

Create a container with iFrame

iFrame is a very old technology that everyone feels ordinary, but it has always worked.

HTML Inline Framework Elements <iframe> represents a nested context being browsed that effectively embeds another HTML page into the current page.

Iframes can create a completely new, stand-alone hosting environment, which means our front-end applications can run independently of each other. There are several important prerequisites for using an iframe:

Website does not require SEO support
Have the appropriate application management mechanism.

If we are working on an application platform, we will integrate a third-party system in our system, or a system under a number of different department teams. Obviously this is a good solution. Some typical scenarios, such as traditional desktop applications, are migrated to web applications:

If this type of application is too complex, then it must be a split for microservices. Therefore, when using an iframe, we need to do two things:

Design Management Application Mechanism
Design Application Communication Mechanism

Load mechanism. Under what circumstances, we will load and unload these applications; in the process, what kind of animation transition is used to make the user look more natural.

Communication mechanism. Creating a postMessage event directly in each app and listening is not a friendly thing. It's inherently intrusive to the application, so getting the Window object of the iFrame element through iframeEl.contentWindow is a much simpler approach. Then, you need to define a set of communication specifications: what format the event name uses, when to start listening for events, and so on.

Interested readers can look at the micro front-end framework that I wrote before: Mooa.

Either way, iframe is afraid that we will not bring benefit to our KPI this year, so let's build a wheel. :)

Homemade micro-frontends framework

Whether it's Web Components-based Angular or VirtualDOM's React, existing front-end frameworks are inseparable from the basic HTML element DOM.

Well, we only need to:

Introduce or create a DOM where appropriate on the page
When the user operates, load the corresponding application (trigger the launch of the application) and uninstall the application.

The first problem, creating a DOM is an easy problem to solve. The second problem is not easy at all, especially to remove the monitoring of the DOM and the corresponding application. When we have a different technology stack, we need to design a set of such logic.

Although Single-SPA already has startup and uninstallation processing for most frameworks (such as React, Angular, Vue, etc.), it is still not suitable for production. When I designed a micro front-end architecture application for the Angular framework based on Single-SPA, I finally chose to rewrite my own framework, Mooa.

Although the difficulty of getting started in this way is relatively high, it is convenient to order and maintainability later. Regardless of the user experience issues caused by each application being loaded, the only possible risk may be: third-party libraries are not compatible.

However, no matter what, compared with iFrame, it is technically more savvy, and more interesting. Similarly, similar to iframes, we still face a series of minor problems:

Need to design a mechanism to manage the application.
For toC applications with high traffic, there will be a lot of requests when loading for the first time.

And we have to split the application again, and want to blabla..., what else can we do?

Combined integration: Widging applications

Combined integration, which is a step-by-step splitting and recombination of the application in the steps of pre-build, build-time, post-build, etc. by means of software engineering.

From this definition point of view, it may not be a micro-frontends - it can satisfy the three elements of the micro-frontends, namely: independent run, independent development, independent deploy. However, with the Lazyload function of the component of the front-end framework - that is, when the required business component or application is loaded, it looks like a micro front-end application.

At the same time, CSS styles don't need to be reloaded because all the dependencies and pollyfill have been loaded as much as possible for the first time.

Common ways are:

Build components and applications independently, generate chunk files, build and then categorize the generated chunk files. (This approach is more similar to microservices, but at a higher cost)
Develop components or applications independently at development time, merge components and applications when integrating, and finally generate single-body applications.
At runtime, load the application's Runtime and then load the corresponding application code and template.

The relationship between the applications is shown in the following figure

This approach seems quite ideal, that is, to meet the parallel development of multiple teams, but also to build a suitable deliverable.

But first, it has a serious limitation: must use the same framework. For most teams, this is not a problem. Teams that use microservices will not use different languages and technologies to develop them because of the front end of microservices. Of course, if you want to use another framework, it is not a problem, we only need to combine the ** homemade framework compatible application in the previous step to meet our needs.

Second, there is a limit to this approach, which is: specification! specification! specification!. In adopting this approach, we need to:

Unified dependencies. Keep these dependent versions and add new ones.
Specification of the components and routes of the application. Avoid conflicts between different applications because these component names conflict.
Build complex. In some scenarios, we need to modify the build system, and in some scenarios we need complex schema scripts.
Share common code. This is obviously a problem that we must face frequently.
Develop code specifications.

Therefore, this approach looks more like a software engineering problem.

Now, we have four options, each with its own pros and cons. Obviously, combining them would be a more ideal approach.

Taking into account the limitations of existing and commonly used technologies, let us look again in the long run.

Pure Web Components technology build

In the process of learning Web Components to develop a micro-frontends architecture, I tried to write my own Web Components framework: oan. After adding some basic web front-end framework features, I found this technology to be particularly suitable for as the cornerstone of the micro-frontends.

Web Components is a different set of technologies that allow you to create reusable custom elements (their functionality is packaged outside of your code) and use them in your web applications.

It consists mainly of four technical components:

Custom elements, allowing developers to create custom elements such as .
Shadow DOM, the shadow DOM, usually attaches the Shadow DOM to the main document DOM and controls its associated functionality. This Shadow DOM cannot be directly controlled by other main document DOMs.
HTML templates, the <template> and <slot> elements, are used to write markup templates that are not displayed on the page.
HTML Imports for introducing custom components.

Each component is introduced by the link tag:



<link rel="import" href="components/di-li.html">
<link rel="import" href="components/d-header.html">

Then, in the respective HTML file, create the corresponding component elements and write the corresponding component logic. A typical Web Components application architecture is shown below:

You can see that this is similar to the way we use iframes above. The components have their own separate scripts and styles, and the corresponding domain names for the individual deployment components. However, it is not as good as it is supposed to be. It is difficult to build front-end applications directly using Web Components:

Rewrite existing front-end applications. Yes, now we need to complete the use of Web Components to complete the functionality of the entire system.
The upstream and downstream ecosystems are not perfect. There is a lack of support for some third-party controls, which is why jQuery is quite popular.
The system architecture is complex. When an application is split into one component after another, communication between components becomes a particularly big problem.

ShadowDOM in Web Components is more like a new generation of front-end DOM containers. Unfortunately, not all browsers can fully support Web Components.

Build with Web Components

Web Components are too far away from us, but combining Web Components to build front-end applications is an architecture for future evolution. Or in the future, we can start to build our application in this way. Fortunately, there are already frameworks to create this possibility.

For now, there are two ways to build a micro front-end application with Web Components:

Build framework-independent components using Web Components and then introduce them in the corresponding framework
Introducing an existing framework in Web Components, similar to the form of an iframe

The former is a component-based approach, or it is like migrating future “legacy systems” to future architectures.

Integrating existing frameworks in Web Components

Existing Web frameworks already have forms that support Web Components, such as createCustomElement supported by Angular, to implement a component in the form of a Web Component:



platformBrowser()
  .bootstrapModuleFactory(MyPopupModuleNgFactory)
    .then(({injector}) => {
      const MyPopupElement = createCustomElement(MyPopup, {injector});
      customElements.define(‘my-popup’, MyPopupElement);
});

In the future, there will be more frameworks that can be integrated into the Web Components application using a form like this.

Web Components integrated into existing frameworks

Alternatively, it is similar to the form of Stencil, which builds the component directly into a component in the form of Web Components, and then in the corresponding such as React or Direct reference in Angular.

Here's an example of a Web Component generated by reference to Stencil in React:



import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';
import registerServiceWorker from './registerServiceWorker';

import 'test-components/testcomponents';

ReactDOM.render(<App />, document.getElementById('root'));
registerServiceWorker();

In this case, we can build a framework-independent component.

The same Stencil still only supports recent browsers such as Chrome, Safari, Firefox, Edge and IE11.

Compound type micro-frontends

Composite type, right in the above categories, just pick several combinations together.

I am not nonsense :)

Microfrontend Quick Selection Guide

I still give the conclusion directly:

The relevant explanations of the key points are as follows:

Frame restrictions. In the back-end microservices system, people use libraries in other languages to develop new services, such as Python for artificial intelligence. But at the front end, there is almost no such possibility. So when we have only one front-end framework, we have a wider range of options when using micro-front-end technology. Unfortunately, most organizations need to be compatible with legacy systems.

IE problem. Whether it was a few years ago or this year, the first consideration we implemented in the micro front end was support for IE. In the projects I have encountered, basically I need to support IE, so there are certain restrictions on the technical selection. And on our projects that don't need to support IE, they can use WebComponents technology to build micro-front-end applications.

Standalone Independence. That is, the dependencies of each micro front-end application are to be managed in a unified manner, or to be managed by themselves in each application. Unified management can solve the problem of repeated load dependencies, and independent management brings additional traffic overhead and latency.

Comparison of micro front-end solutions: a brief comparison

If you are still not familiar with the above aspects, please read the "Six and Seven Ways of Implementing Front-End Microservices."

Ways	Development Costs	Maintenance Costs	Feasibility	Same Framework Requirements	Difficulties in Implementation
Route Distribution	Low	Low	High	No	★
iFrame	Low	Low	High	No	★
Application Microservices	High	Low	Medium	No	★★★★
Micro-Widget	High	Medium	Low	Yes	★★★★★
Micro-apps	Medium	Medium	High	Yes	★★★
Pure Web Components	High	Low	High	No	★★
Web Components	High	Low	High	No	★★

Similarly, some complex concepts are explained as follows:

Application micro-services, that is, each front-end application is a separate service-oriented front-end application, and is equipped with a unified application management and startup mechanism, such as micro-front-end framework Single-SPA or mooa.

Micro-Widget, that is, through the hack of the build system, different front-end applications can use the same set of dependencies. It basically improves the problem of repeatedly loading dependent files in Apps Microservices.

Micro-apps, also known as combined integration, that is, through software engineering, splitting single applications in the development environment, and combining the applications into one in the build environment application. For detailed details, you can look forward to the following article "Disassembly and Micro-Service of a Single Front-End Application"

Comparison of micro front-end solutions: complex ways

I saw a micro-service related article, introduces the difference between different microservices, which uses a more interesting comparison method in detail, here is the same way to show:

Architectural goal	description
a. Independent development	Independent development without being affected
b. Standalone deployment	Can be deployed as a single service
c. Support different frameworks	Can use different frameworks at the same time, such as Angular, Vue, React
d. Shake tree optimization	Can eliminate unused code
e. Environmental isolation	The context between applications is undisturbed
f. Multiple applications running simultaneously	Different applications can run simultaneously
g. Shared dependencies	Whether different applications share the underlying dependency library
h. Dependency conflict	Whether different versions of dependencies cause conflicts
i. Integrated Compilation	The application is finally compiled into a whole, not built separately

Then, for the table below, a~j in the table represent several different architectural considerations above.

(PS: Considering the length of several words of Web Components, temporarily refer to it as WC~~)

Way	a	b	c	d	e	f	g	h
Route Distribution	O	O	O	O	O	O
iFrame	O	O	O	O	O	O
Application Microservices	O	O	O			O
Widget	O	O			-	-	O	-
Micro-application	O	O		O	-	-	O	-
Pure WC	O	O		O	O	O	-	-
Combine WC	O	O	O	O	O	O

O in the figure indicates support, blank indicates no support, and - indicates no effect.

Combine the previous selection guide:

How to deconstruct mono front-end application - microservice splitting of front-end applications

Refresh the page? Route splitting? No, dynamically load components.

This article is divided into the following four parts:

Introduction to front-end micro-services
Micro front end design concept
Practical micro front-end architecture design
Front-end microservices based on Mooa

front end microservice

For front-end microservices, there are some options:

Web Component obviously has a very good infrastructure. However, we are not likely to rewrite existing applications in large numbers.
iFrame. Are you serious?
Another micro front-end framework, Single-SPA, is obviously a better way. However, it is not Production Ready.
Split the application by routing, and this jump will affect the user experience.
and many more.

Therefore, when we consider front-end micro-services, we hope that:

Independent deployment
Independent development
technology independently
Does not affect the user experience

Independent development

In the past few weeks, I spent a lot of time learning the code for Single-SPA. However, I found it really too cumbersome to develop and deploy, and I couldn't reach the standard of independent deployment. According to the design of Single-SPA, I need to name my application in the entry file before I can build it:



declareChildApplication('inferno', () => import('src/inferno/inferno.app.js'), pathPrefix('/inferno'));

At the same time, in my application, I still need to specify my life cycle. This means that when I develop a new application, I have to update two pieces of code: the main project and the application. At this time we are also very likely to work in the same source.

When there are multiple teams working in the same source, it obviously becomes quite unreliable - for example, the other team is using Tab, and we are using 2 spaces, the next door is used by Pharaoh. 4 spaces.

Independent deployment

The biggest problem with a single front-end application is that the js and css files built are quite large. The micro front end means that the file is split into multiple files independently, and they can be deployed independently.

Do we really need technology independently?

Wait, do we really need technology independently? If we don't need technology, the micro front-end problem is easy to solve.

In fact, for most companies and teams, technology has nothing to do with an irrelevant speech. When several founders of a company use Java, it is highly likely that Java will continue to be used in future selections. Unless, there are some extra services to implement artificial intelligence using Python. Therefore, in most cases, it is still the only technology stack.

This is especially true for front-end projects: basically only one framework is selected in a department.

So we chose Angular.

Does not affect the user experience

Using route jumps for front-end micro-services is a very simple and efficient way to split. However, during the route jump, there will be a white screen process. In this process, the application before the jump and the application to be jumped lose control of the page. If there is a problem with this application, then the user will be overwhelmed.

Ideally, it should be controllable.

Micro-frontends design concept

Design Concept 1: Centralized Routing

Is the nature of the Internet decentralized? No, DNS has decided that it is not. FLAG / TAB , decided that it is not.

In essence, microservices should be decentralized. However, it cannot be completely decentralized. For a microservice, it requires a service registry:

The service provider wants to register the notification service address, and the caller of the service should be able to discover the target service.

For a front-end application, this thing is routing.

From the page, only when we add a menu link on the page, the user can know that a page is available.

From the code point of view, that is, we need to have a place to manage our application: find out which applications exist and which application uses which route.

Managing our routes is actually managing our applications.

Design Concept 2: Identification Application

When designing a micro front-end framework, the problem of getting a name for each project lingered me for a long time - how to normalize this thing. Until, I once again thought of Conway's law:

System design (product structure is equivalent to organizational form, the organization of each design system, which produces a design equivalent to the communication structure between organizations.

In other words, it is impossible to have two projects with the same name under the same organization.

Therefore, this problem is solved simply.

Design Concept 3: Life Cycle

Single-SPA has designed a basic lifecycle (although it is not managed uniformly) and it contains five states:

load, decide which application to load and bind the lifecycle
bootstrap, get static resources
mount, install the application, such as creating a DOM node
unload, delete the life cycle of the application
unmount, uninstall the application, such as deleting the DOM node

So, I basically followed this life cycle in design. Obviously, things like load are superfluous to my design.

Design Concept 4: Independent Deployment and Configuration Automation

In a sense, the entire system is built around the application configuration. If the configuration of the application can be automated, the entire system is automated.

When we only develop a new component, then we only need to update our components and update the configuration. And this configuration itself should also be automatically generated.

Practical micro-frontends architecture design

Based on the above premise, the workflow of the system is as follows:

The overall engineering process is as follows:

When the main project is running, it will go to the server to get the latest application configuration.
After the main project gets the configuration, it will create the application one by one and bind the lifecycle to the application.
When the main project detects the route change, it will find out if there is a corresponding route matching to the application.
When the matching pair corresponds to the application, the corresponding application is loaded.

Therefore, its corresponding structure is shown below:

The overall process is shown below:

Independent Deployment and Configuration Automation

The deployment strategy we made is as follows: The configuration file used by our application is called apps.json, which is obtained by the main project. Every time we deploy, we just need to point apps.json to the latest configuration file. The configured file class is as follows:

96a7907e5488b6bb.json
6ff3bfaaa2cd39ea.json
dcd074685c97ab9b.json

The configuration of an application is as follows:



{
  "name": "help",
  "selector": "help-root",
  "baseScriptUrl": "/assets/help",
  "styles": [
    "styles.bundle.css"
  ],
  "prefix": "help",
  "scripts": [
    "inline.bundle.js",
    "polyfills.bundle.js",
    "main.bundle.js"
  ]
}

Here selector corresponds to the DOM node required by the application, and prefix is used for URL routing. These are automatically generated from the index.html file and package.json.

Inter-Application Routing - Events

Because the current application has become two parts: the main project and the application part. There will be a problem: Only one project can capture routing changes. When the primary route of the application is changed by the main project, it cannot be effectively communicated to the sub-application. At this time, the sub-application can only be notified by means of an event, and the sub-application also needs to monitor whether it is the route of the current application.



if (event.detail.app.name === appName) {
  let urlPrefix = 'app'
  if (urlPrefix) {
    urlPrefix = `/${window.mooa.option.urlPrefix}/`
  }
  router.navigate([event.detail.url.replace(urlPrefix + appName, '')])
}

Similarly, when we need to jump from application A to application B, we also need a mechanism like this:



window.addEventListener('mooa.routing.navigate', function(event: CustomEvent) {
  Const opts = event.detail
  If (opts) {
    navigateAppByName(opts)
  }
});

The rest of the animations like Loading are similar.

Four to split large Angular application to micro-frontends

In 2018, we spent a lot of time designing a solution to split a large Angular app. A series of discussions took place from the use of Angular's Lazyload to front-end microservices. Finally, we finally got the result, using the Lazyload variant: the way to integrate code when building.

As a “professional” consultant, I have been busy designing an Angular split service solution for my clients. Mainly to achieve the following design goals:

Build a plug-in web development platform to meet the needs of rapid business change and distributed multi-team parallel development
Build serviced middleware to build a highly available and highly multiplexed front-end microservice platform
Support independent delivery and deployment of front ends

Simply put, it is to support application plug-in development, and multi-team parallel development.

Application plug-in development, the main problem to be solved is: the split problem of bloated large-scale applications. Large front-end applications face a lot of ** legacy code when developing, and code of different services are coupled together. When uploading online, they also face slow loading and low operating efficiency.

Finally, it falls on two schemes: routing lazy loading and its variants and front-end micro-services

1, front-end micro-services: routing lazy loading and its variants

The route is lazy loaded, that is, the application is cut into different codes through different routes, and the corresponding component is loaded when the route is accessed. In the framework of Angular and Vue, it can be implemented by routing + Webpack. And, inevitably, some problems will be needed:

It is difficult to develop multiple teams in parallel. Route splitting means that we still work in a source repository. You can also try splitting into different projects and compiling them together.

Every release needs to be recompiled, yes, when we just update the code of a submodule, we have to recompile the entire application and republish the application. Instead of building it independently, release it.

Unified Vendor version, it is a good thing to unify third-party dependencies. The key to the problem is that whenever we add a new dependency, we might need to have a meeting to discuss it.

However, the biggest problem with the standard Route Lazyload is that it is difficult to develop multiple teams in parallel. The reason why it is said is “difficult” because there is still a way to solve this problem. In daily development, a small team will always be developed in a code base, while a large team should be developed in a different code base.

So, we did some experiments on the standard route lazy loading.

For a team of 20 or 30 people, they may belong to different departments in the business, and there are technically inconsistent specifications, such as 4 spaces, 2 spaces or Tab. Especially when it is a different company and team, they may have to abandon a series of questions such as testing, code static detection, code style unification and so on.

2, microservice solution: sub-application mode

In addition to routing lazy loading, we can also use the sub-application mode, that is, each application is independent of each other. That is, we have a pedestal project. When the user clicks on the corresponding route, we load the ** independent ** Angular application; if it is the route under the same application, it does not need to be reloaded. Moreover, these can all be done by relying on the browser cache.

In addition to routing lazy loading, you can also use an application embedding solution similar to Mooa. The following is an example of HTML generated based on the Mooa framework + Angular development:



<app-root _nghost-c0="" ng-version="4.2.0">
  ...
  <app-home _nghost-c2="">
    <app-app1 _nghost-c0="" ng-version="5.2.8" style="display: none;"><nav _ngcontent-c0="" class="navbar"></app-app1>
    <iframe frameborder="" width="100%" height="100%" src="http://localhost:4200/app/help/homeassets/iframe.html" id="help_206547"></iframe>
  </app-home>
</app-root>

Mooa provides two modes, one is based on Single-SPA experiments, loading and rendering two Angular applications on the same page; one is based on iFrame to provide a separate application container.

Solved the following problems:

Home page loading faster, because only the features needed for the home page need to be loaded, not all dependencies.
Multiple teams are developing in parallel, each team can be independently developed in their own projects.
Modular update independently, now we only need to update our application separately without having to update the entire complete application.

However, it still contains the following issues:

Repeated loading of dependencies, ie the modules we use in the A app, will also be reused in the B app. Some can be solved automatically by the browser's cache.
It takes time to open the corresponding app for the first time, of course ** preloading** can solve part of the problem.
Running in non-iframe mode, you will encounter unpredictable third-party dependency conflicts.

So after summarizing a series of discussions, we formed a series of comparisons:

Solution comparison

In this process, we have done a lot of program design and comparison, and I want to write an article to compare the previous results. Look at the picture first:

Table comparison:

x	Standard Lazyload	Build-Time Integration	Post-Build Integration	Application Independent
Development process	Multiple teams are developing in the same code base	Multiple teams are developing in different code bases	Multiple teams are developing in different code bases	Multiple teams are developing in different code bases
Build and publish	Build only need to take this code to build, deploy	integrate the code of different code bases, and then build the application	will be compiled directly into each project module, the runtime will be merged through lazy loading	will be compiled directly Into several different applications, the runtime is loaded by the main project
Applicable scenarios	Single team, less dependent library, single business	Multiple teams, less dependent libraries, single business	Multiple teams, less dependent libraries, single business	Multiple teams, relying on multiple systems, complex business
Performance mode	Development, construction, operation integration	Development separation, construction integration, operation integration	development separation, construction separation, operation integration	development, construction, operation separation

The detailed introduction is as follows:

Standard LazyLoad

Development process: Multiple teams are developed in the same code base, and only need to use this code to deploy.

Behavior: development, construction, and operation

Applicable scenarios: single team, less dependent libraries, single business

LazyLoad Variant 1: Build-time integration

Development process: Multiple teams are developed in different code bases. When building, integrate the code of different code bases and build the application.

Applicable scenarios: multiple teams, less dependent libraries, single business

Variant-build integration: development separation, integration when building, running one

LazyLoad variant 2: Post-build integration

Development process: Multiple teams are developed in different code bases, compiled into different pieces of code at build time, and merged together by lazy loading.