DEV Community: Yisar

Chinese miniapp architecture

Yisar — Thu, 08 Jun 2023 05:25:15 +0000

Do you know that Chinese people have their own cross platform framework, called miniapp? As of now, there are approximately 7 million miniapps, and 1.4 billion Chinese people can avoid using browsers and search engines, but nobody can avoid using miniapps.

I have introduced this architecture to few foreign companies and developers before.

As far as I know, @elonmusk is very envious of WeChat's payment, But he didn't know that without WeChat miniapps, all fetures would not be universal.

I used to work at Trip.com, leading the development of Trip miniapp. Now that I have resigned, I no longer do front-end work, but I don't want to be forgotten, so I have openesourced this architecture.

China is a vast internet world, its technology should not be closed source.

Screenshots

Github

https://github.com/yisar/smallapp

Asta: New jointing and Resumable SSR framework.

Yisar — Sat, 22 Oct 2022 04:15:46 +0000

Hello everyone, I am Yisar, a Javascript developer from China.

In China, there are few web pages, but just three months ago, I joined a game company. We have foreign websites and some foreign users.

I led our team to start the SSR transformation of our website, but finally failed because the QPS is too low.

This is because the SSR isomorphism requires the generation and traversal of vdom, and the server side cannot afford such computation.

No matter vue or react, or even preact or svelte, they all face the same difficulties.

Is there really no way？

Asta.js

Finally, I decided to write a new framework to optimize the characteristics of SSR, with the goal of solving the performance bottleneck of SSR.

It is meaningless to discuss performance beyond bottlenecks. If we want to optimize, we should start with performance bottlenecks

Server side bottlenecks, Jointing on Server

The biggest performance bottleneck of SSR lies in the server side. The server is very fragile. The traditional homogeneous SSR framework requires the generation and traversal of vdom, which brings pressure on the CPU. If the experience is not enough, it is also easy to leak memory.

To solve this problem, Asta has a jsx compiler, which no longer generates vdom, but now generates s functions for string splicing.

// jsx input
const view = ({list}) => <div>{list.map(i=><i>{i}</i>)}</div>
// server output
const view = ({list}) => s.openTag('div')+s.expression(list.map(i=>s.openTag('i')+s.text(i)+s.closeTag('i')))+s.closeTag('div')

It does not need babel, and perfectly retains the semantics of js.

It is worth mentioning that this is also completely feasible at the data structure level. All tree structures can be expressed through linear structures.

The second performance bottleneck on the server side is the component itself. The initialization, private state (usually publish and subscribe), and lifecycle of the component are huge performance overhead and difficult to be optimized by the compiler.

To solve this problem, Asta uses Elm's state flow model. The component itself has no state, and all states flow from the global single state tree from top to bottom.

export const loader = async (req) => {
    const data = await fetch('http://localhost:1234/data')
    .then((res) => res.json())
      .then((data) => data)
    return {
        title: data.title,
    }
}
const Header = ({ title }) => (
    <header>
    <h1>{title}</h1>
    </header>
)

export default ({ title }) => {
    return (
    <div>
        <Header title={title}/>
    <div>
    )
}

Yes, components are also pure functions, which are only used to splice strings without any overhead.

Pressure test results.

After these two optimizations, the performance of asta is super good. We pressure tested a simple application with an interface in the company, and the results are as follows.

Because Vue3 has the template compilation optimization of block tree, its QPS is higher than nextjs, but it still cannot avoid component overhead.

Asta's QPS is amazing. It's even much higher than Markojs.

Resumable on client

We have solved the performance bottleneck on the server side. We begin to focus on the client side. The performance cost of traditional SSR lies in full hydration.

A new concept named Resumable comes from Qwik.js, its basic idea is not to hydrate, serialize the necessary information into html, and then use vdom and dom on the client side to diff to directly recover the information.

Resumable

It is very suitable for the mental model of asta, so I implemented it easily.

In addition, in order to reduce the size of the client side js, people have also come up with other methods, such as selective hydration, islands, server components. However, the mental models are too corrupt, so we won't discuss them one by one.

Resumable is better at 0 js and Google scoring.

Summary

To sum up, after double optimization, asta has no bottleneck in its performance. It is open source. The github address is here.

https://github.com/yisar/asta

If you are interested, you can run the demo to try, and you will understand.

I am a Chinese, so I may not use English all the time, but I also welcome exchanges.

2021 JavaScript framework

Yisar — Mon, 23 Aug 2021 03:45:32 +0000

2021 is very different from previous years. Many web frameworks have released their new versions. Let's discuss them.

Vdom-based

For example, React, Vue and Preact all belong to this category.

This is also the most mature framework. Their advantages are mature, stable and suitable for most scenarios.

With SSR and partial hydration, react 18's fizz renderer is very interesting.

In addition, such as concurrent mode, scheduling optimization has been discussed for many years.

Fre is a little soldier in this camp. It is the smallest but the fastest.

https://github.com/yisar/fre

Compilation instead of runtime

Svelte, solidjs belongs to this category.

I always think that the web does not need to be optimized to the extreme, but the code architecture needs to be pursued unilaterally.

So I prefer Svelte a bit. Compilation and runtime should not coexist, which will make the framework more messy and mediocre.

Solidjs and Vue3 do not do this well. They have compilation and want to runtime. They can't have both fish and bear's paw.

Html first

History is reversing，recently, some frameworks want to return to HTML, away from vdom and away from compilation.

Such as Qwikjs and Alpinejs. They are enhancing HTML, Sprinkling intead of Hydration.

This does fill a part of the cake, but the performance will be lost. For example, alpine has always ranked last in the benchmark.

Summary

I like all three ideas. How about you?

Using one of them is entirely determined by my work.

I never need webpack or babel anymore

Yisar — Wed, 28 Jul 2021 02:39:06 +0000

In the past period of time, I was refactoring our company’s miniapp compiler, Its performance has increased by more than ten times.

Use Esbuild

Replace webpack or rollup

What's interesting, using esbuild and webpack are almost the same.

webpack node api

const webpack = require('webpack')

webpack({
  entry: './src/index.js',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: '[name].js',
  }
}).run((err, stats) => {
})

esbuild node api

const esbuild = require('esbuild')

await esbuild.build({
  entryPoints: [path.join(cwd, 'src', 'app.js')],
  outdir: path.join(cwd, 'dist'),
  bundle: true
})

They are no different in use, but esbuild is much faster than webpack.

In most places where you use webpack, you can use esbuild instead.

Replace jest

In my open source framework fre, I use playwright + zora + esbuild as the test toolkits, and its experience is very smooth.

https://github.com/yisar/fre/tree/master/test

before:
puppeteer + jest 30s+

after:
playwright + zora + esbuild 3s-

Replace terser

https://github.com/privatenumber/minification-benchmarks

Using esbuild as a compressor for large projects can improve a lot of performance.

Replace next.js

We have a new alternative within our company, which uses esbuild throughout the toolchain, and the development experience is much better than next.js.

Disadvantages of esbuild

AST operation is not supported
Generating d.ts is not supported
No go API provided
The generated runtime code is dirty

After we use esbuild in depth, we find it has many disadvantages, so it is impossible to use esbuild alone.

Some people use rollup, such as the vite team, but I don't think it's worth it.

Use Swc

This is another tool written using rust, which can make up for some shortcomings of esbuild.

Replace babel

A common practice is to modify AST in rust side, and then pass the results to node side through wasm.

https://github.com/parcel-bundler/parcel/blob/v2/packages/transformers/js/core/src/hoist.rs

For example, that's what parcel v2 does.

Summary

Most of the time, esbuild is enough. swc can be used when the AST needs to be modified, but anyway, I don't need webpack or babel anymore.

More and more similar tools will appear in the front-end tool chain in the future. They are written in go or rust, and they are very fast and smooth.

I love this change very much.

Fre-2.1 has been pulished

Yisar — Fri, 11 Jun 2021 06:10:36 +0000

I announce that fre2 is officially released, which is a major breakthrough version.

Offscreen rendering

The bigist breakthrough is offscreen rendering, a core algorithm refactor.

https://github.com/yisar/fre/releases/tag/2.1.0-alpha

before:

after:

Offscreen rendering is an algorithm level optimization, it traverses vdom in reverse order, from bottom to top, from right to left, to ensure that the front DOM pointer is in memory, and finally it is drawn to the screen at one time.

With off screen rendering, fre has become the best performance frameworks in vdom world, not one of.

Just as react 18 also released alpha version, fre 2.1 also tried to be compatible with them.

CreateRoot

import { render, useState } from 'fre'

function App() {
  const [count, setCount] = useState(0)
  return <>
      <h1>{count}</h1>
      <button onClick={() => setCount(count + 1)}>+</button>
    </>
}

createRoot(document.body).render(<App/>) // here

This API is more ergonomic, and for the callback, you can do this:

function App({ callback }) {
  return (
    <div ref={callback}>
      <h1>Hello World</h1>
    </div>
  );
}

createRoot(document.body).render(<App callback={() => console.log("renderered")} />)

startTransiton

This is an API for lowering priority, which is very useful, so I decide to build it in.

function App() {
  const [count, setCount] = useState(0)

  console.log(count) // 1 2

  const update = () =>{
    startTransition(()=>{
      setCount(2)
    })
    setCount(1)
  }
  return <>
      <h1>{count}</h1>
      <button onClick={update}>+</button>
    </>
}

It works can be understood as setTimeout (cb, 0), but the callback function is executed synchronously, and the update is delayed asynchronously.

auto-updates

https://github.com/yisar/fre/blob/master/demo/src/auto-batch.tsx

Fre has always been supportive.

Suspense SSR

This is the only breakthrough of react 18. I like it very much, but fre doesn't support it now.

I need to spend some time to study it.

Summary

Fre2 has also been released. If you are interested in the front-end framework, you can jump to GitHub.

https://github.com/yisar/fre

It has all the advanced features of react 18, but only 400 lines of code, and its performance is much better than react.

Fre Offscreen rendering: The fastest vdom algorithm

Yisar — Tue, 08 Jun 2021 03:24:53 +0000

In the past, I wrote a front-end framework named fre in school, which uses the data structure of fiber linked list.

In fre v1, the biggest breakthrough is Time slicing. It can greatly improve the response performance.

Now that fre2 is released, I've made a breakthrough again.

It is called Offscreen rendering. It operates DOM in memory and draws them to the screen once at the last time.

In the actual benchmark test, Fre is the fastest, it broke the rules of the game.

In the past, we always thought that frameworks with vdom could not be faster than vanilla JS, while svelte's idea could be close to the performance of vanilla JS.

https://github.com/yisar/fre/blob/master/demo/src/benchmark.tsx

Fre with off-screen rendering breaks this fact. I'm proud to say that fre is the fastest vdom-based framework, and it's only 400 lines, the smallest size.

It is worth mentioning that this is a highly optimized data structure and core algorithm.

And it does not conflict with time slicing and does not depend on the 'DocumentFragment' API.

In fact, similar optimizations always exist in IOS or OpenGL. It's not easy for web to do this, so fre is the first framework to implement it.

I don't want to introduce too many implementation details of off-screen rendering. If you are interested in this, you can reply to it or read the source code of fre.

https://github.com/yisar/fre

I really hope to get your feedback, because I am a Chinese, usually in school or company, no one can discuss these with me, I can only study alone.

Thank you very much!

Use isInputPending API for better scheduler

Yisar — Tue, 02 Feb 2021 03:00:34 +0000

As you know, fre is an asynchronous rendering UI library that similar to react. The rendering process of components can be interrupted

The priority scheduling of react first enumerates and classifies events, which is too much code.

How can fre implement priority without enumerating event names?

isInputPending()

Inspired by react fiber, Facebook has written a proposal that when the browser is in the input pending state, you can interrupt your JS loop

while(navigator.scheduling.isInputPending()){
  setTimeout(reconcileWork) open a new tick
}

We also can use this API to divide priorities, such as this:

dom.addEventListener('input', (e) => {
  let isHighPriority = navigator.scheduling.isInputPending()
  setState(count + 1, isHighPriority)
})

It's done. We have a priority system with just one API.

https://github.com/yisar/fre/pull/226

At present, it has been shipped in fre, and we will do more based on it in the future.

It's worth mentioning that fre now has three priorities

high priority => input pending => immediately
common priority => time slicing => 16ms
low priority => component update => It's been interrupted

If you are interested in asynchronous rendering or react internal implementation principles, welcome to fre's GitHub

https://github.com/yisar/fre

Update granularity of Javascripts frameworks

Yisar — Sun, 31 Jan 2021 11:38:19 +0000

Do you know that the update granularity of different Javascript frameworks is different? Let's talk about it in detail.

Coarse grained

Almost all the framework updates based on vdom are coarse-grained. The unit they update is component, and then the whole component will be re rendered for reconciliation.

Such as react, fre……

There are the advantages of vdom, but it will cause unnecessary updates. The react team says that the performance of JS runtime is negligible, so the coarse-grained update of vdom has always been a good way.

I agree with this option, because if it is not for coarse-grained update, asynchronous rendering will not be easy, and the essence of concurrent mode is repeated parallel rendering.

Fine grained

Some proxy based frameworks are fine-grained. Such as svelte, vue1. They hijack the keys of state and can update them accurately.

This is a good idea most of the time, but when there is a long list, too many proxies will be generated, and the performance will be degraded.

Block grained

This is a compromise, that is, when a block is encountered, the runtime is used for reconciliation.

Such as vue3:

<div v-for="item in items" key="item.key"/>

output like this:

for(items, (item)=> item.key)

The for function is a runtime function that performs internal reconciliation.

trade off

Three different granularity are good and bad, as a framework, the author should weigh the choice.

Fre uses vdom, so its updates are coarse-grained, which is more conducive to asynchronous rendering.

But I think that for frameworks like svelte, using block granularity is the best, because many algorithms are runtime optimizations.

https://github.com/yisar/fre

Finally, if you are interested in the new framework, take a look at fre.

Start using time slicing through fre.js

Yisar — Sat, 09 Jan 2021 11:43:26 +0000

Do you know time slicing?

This is an amazing thing in react concurrent mode, which shows that the browser will not be blocked by component rendering.

import { h, render, useState, useEffect } from 'fre'

const UPDATE_EVERY = 500
const BLOCK_FOR = 5
const NUM_COMPONENTS = 100

const App = () => {
  const [count, setCount] = useState(0)

  useEffect(() => {
    setTimeout(() => setCount(count + 1), UPDATE_EVERY)
  })

  const values = []

  for (let i = count; i < count + NUM_COMPONENTS; i++) {
    values.push(i)
  }

  return (
    <div className="wraper">
      <h1>Count: {count}</h1>
      {values.map((value, index) => (
        <SlowComponent key={value} value={value} />
      ))}
    </div>
  )
}

const SlowComponent = ({ value }) => {
  const start = performance.now()
  while (performance.now() - start < BLOCK_FOR) {}

  return <li className="slow">{value}</li>
}

const root = document.getElementById('root')
let div = document.createElement('div')
div.innerHTML = `<style>
body {
  background: #010911;
  font-family: sans-serif;
}
h1 {
  color: rgb(0, 204, 10);
}
#animation {
  width: 380px;
  height: 380px;
  transform: translate(50%, 50%);
  margin: 20px auto;
}
.slow {
  background: #f60066;
  color: white;
  padding: 5px;
  display: inline-block;
  margin: 5px;
  font-size: 12px;
  border-radius: 50%;
}
.wraper{
  margin: 20px auto;
  width: 700px;
  text-align: center;
}
ul.solarsystem {
  position: relative;
  height: 400px;
  list-style: none;
  transition: all 0.09s ease-in;
  transform: translate(-480px, -336px);
}
ul.solarsystem li {
  text-indent: -9999px;
  display: block;
  position: absolute;
  border: 2px solid #394057;
}
ul.solarsystem li span {
  display: block;
  position: absolute;
  width: 10px;
  height: 10px;
  border-radius: 5px;
}
ul.solarsystem li.active.sun span,
ul.solarsystem li.active.earth .moon {
  border: none;
  box-shadow: none;
}
ul.solarsystem li.sun {
  width: 40px;
  height: 40px;
  border-radius: 20px;
  background: #fc3;
  background-image: gradient(linear,
      left bottom,
      left top,
      color-stop(0.22, rgb(204, 153, 0)),
      color-stop(1, rgb(255, 219, 112)));
  top: 302px;
  left: 462px;
  border: none;
  box-shadow: 0 0 50px #c90;
  z-index: 100;
  transition: all 0.2s ease-in;
}
ul.solarsystem li.sun span {
  width: 60px;
  height: 60px;
  border-radius: 50%;
}
ul.solarsystem li.mercury {
  width: 100px;
  height: 100px;
  border-radius: 52px;
  top: 270px;
  left: 430px;
  z-index: 99;
}
ul.solarsystem li.mercury span {
  background: #b6bac5;
  top: 10px;
  left: 10px;
}
ul.solarsystem li.venus {
  width: 160px;
  height: 160px;
  border-radius: 82px;
  top: 240px;
  left: 400px;
  z-index: 98;
}
ul.solarsystem li.venus span {
  background: #bf8639;
  top: 118px;
  left: 5px;
}
ul.solarsystem li.earth {
  width: 220px;
  height: 220px;
  border-radius: 112px;
  top: 210px;
  left: 370px;
  z-index: 97;
}
ul.solarsystem li.earth span {
  background: #06c;
  top: 56px;
  left: 5px;
}
ul.solarsystem li.earth span.moon {
  width: 4px;
  height: 4px;
  border-radius: 2px;
  background: #ccc;
  top: 12px;
  left: 12px;
}
ul.solarsystem li.mars {
  width: 280px;
  height: 280px;
  border-radius: 142px;
  top: 180px;
  left: 340px;
  z-index: 96;
}
ul.solarsystem li.mars span {
  background: #aa4200;
  top: 0px;
  left: 175px;
}
ul.solarsystem li.asteroids_meteorids {
  top: 155px;
  left: 315px;
  z-index: 1;
  width: 330px;
  height: 330px;
  border-radius: 165px;
  border: none;
}
ul.solarsystem li.jupiter {
  width: 340px;
  height: 340px;
  border-radius: 172px;
  top: 150px;
  left: 310px;
  z-index: 95;
}
ul.solarsystem li.jupiter span {
  background: #e0ae6f;
  top: 67px;
  left: 24px;
}
ul.solarsystem li,
ul.solarsystem li.earth span {
  animation-iteration-count: infinite;
  animation-timing-function: linear;
  animation-name: orbit;
}
ul.solarsystem li.mercury {
  animation-duration: 5s;
}
ul.solarsystem li.venus {
  animation-name: retrograde;
  animation-duration: 8s;
}
ul.solarsystem li.earth {
  animation-duration: 12s;
}
ul.solarsystem li.earth span {
  animation-duration: 2s;
}
ul.solarsystem li.mars {
  animation-duration: 20s;
}
ul.solarsystem li.asteroids_meteorids {
  animation-duration: 50s;
}
ul.solarsystem li.jupiter {
  animation-duration: 30s;
}
@keyframes orbit {
  from {
    transform: rotate(0deg);
    color: white;
    font-size: 200%;
  }
  to {
    transform: rotate(360deg);
    color: black;
    font-size: 50%;
  }
}
@keyframes retrograde {
  from {
    transform: rotate(0deg);
    color: white;
    font-size: 200%;
  }
  to {
    transform: rotate(-360deg);
    color: black;
    font-size: 50%;
  }
}
</style>
<div id="animation">
<ul class="solarsystem">
  <li class="sun"><span>Sun</span></li>
  <li class="mercury"><span>Mercury</span></li>
  <li class="venus"><span>Venus</span></li>
  <li class="earth"><span>Earth<span class="moon">Moon</span></span>
  </li>
  <li class="mars"><span>Mars</span></li>
  <li class="asteroids_meteorids"><span>Asteroids</span></li>
  <li class="jupiter"><span>Jupiter</span></li>
</ul>
</div>`

document.body.insertBefore(div, root)
render(<App />, root)

In this demo, CSS animation is running all the time. If you use the synchronous framework such as preact or vue, you will feel obvious stuck jank.

But if you use fre or react, you can keep a consistent response, which is very important.

Why?

As we all know, microtask clears the queue before the browser's layout, while Vue and preact update their components in microtask.

// vue
this.a = 0
this.b = 1
Promise.resolve().then(updateComponent) // atter a b updated

They use the fetures of microtask to achieve batch updates, but also block the browser because of the feature of microtask.

In fre (or react), rendering is asynchronous, which makes use of the feature that macro tasks don't block the browser to do time slicing.

Fre fiber implements a lightweight rendering engine, which uses vdom to generate linked list, interrupts and continues in the traversal of linked list, giving the browser the chance to repaint.

Useful?

We heard a voice that the preact and vue teams don't like time slicing. They think there are very few scenes of time slicing.

But believe me, time slicing is very useful. In addition to animation, it can also do visualization related work.

In addition, do you know the situation of react native, a non web platform? It doesn't have as good performance and memory as web. Without time slicing, animation will be difficult.

Others

In addition to the value of time slice, its research significance is greater. Fiber rendering engine is a new exploration, there are many things to do, I need partners.

https://github.com/yisar/fre

If you have a good idea, please let me know.

Fre: another concurrent UI library

Yisar — Fri, 08 Jan 2021 06:10:33 +0000

Hello everyone, my name is yisar, a front-end enginer in China.

This is my first visit to media, and I want to share my framework with you.

React is very popular, especially after react 16. The most amazing thing is concurrent mode, which is not available in other frameworks.

Fre is the second framework to implement concurrent mode in addition to react. This implementation is different from the other vdom reconciliation algorithm. Its rendering is asynchronous, and many use cases are built on the premise of asynchronous rendering.

Fre is similar to react in many ways, such as fiber architecture using linked list, hooks API and functional component…

But at the same time, they are different. Fre has a better reconciliation algorithm. The most important thing is that fre only has 400 lines of code, the bundle size is just 1KB!

In most use cases, the performance of fre is close to that of vue3 and better than that of react.

At the same time, 1KB of code means that you can better understand the source code of fre, and even learn react with the help from fre.

It is worth mentioning that preact is as small as fre, but preact is completely synchronous. It does not support concurrent mode and keeps the opposite roadmap with react.

If you are looking for an alternative framework of 1KB react, fre is more suitable for you.

If you’re interested in concurrent mode, take a look at fre and you’ll get a ruby.

https://github.com/yisar/fre

I really need partners, because fre has a lot to improve, let’s build together.