DEV Community: Alan

Learning Resources for WebGPU, OpenGL, and Shaders

Alan — Thu, 30 Apr 2020 18:23:14 +0000

In the near future, your browser will be able to run graphic-heavy games, bitcoin mining, and TensorFlow thanks to WebGPU.

Here are beginner-friendly learning resources:

To Those Who Thinks UI is Hard

Alan — Tue, 14 Apr 2020 01:44:57 +0000

This is a short essay for programmers who think designing UI is hard, but find success in designing Rest API, CLI, Custom IPC Protocol, maintainable C header file, etc.

Easiest way to approach any kind of design is to identify the consumer first. For UI it's the user and their task. Measure their capability, what they're good at doing and not, and optimize from it.
Human is capable of short spanned attention, short memory, and pattern matching. They're not really good at / fond of remembering many things at once, imagining big numbers, context switching, doing boring work, and waking up at 6 on holidays.

Once identified, you'll know the part of the task your software should do and the part you can leave with to user. This is the same reason why you most likely won't find an RPG game where the computer moves the main character and the player rolls the dice. Optimize around it.

Hand the user control of what they can do best, your software will be seen as useful. Make your software flexible by opening up a way for user to operate on deeper and more complex layer by hiding them carefully so that it is accessible through the user's careful perception, e.g. the "advanced setting" button. Be transparent to the user about what you're doing with the software, tell the users the complex things your software is doing for them and your software will be educative.

Those are the important thing. Platforms (browser, mobile, html, native, electron, QT, unreal engine) are secondary and frameworks (react, angular, svelte, vue, flutter, bootstrap, bulma css, styled components) are tertiary.

Use these concepts to help users understand what you're showing. Color helps with intention, red is danger, green is positive, blue is secure. Contrast helps with directing user's attention. Transition and animation help with retaining context in user's mind. Spacing helps with separating concerns. Grouping and hierarchy helps with... well grouping, and hierarchy. There's a lot more. Typography, focal vs periphery, shapes, interaction design, information architecture, gestalt etc.

This is the same principle as any other design process. Product design is almost the same in concepts and tooling, with the exception the jargons are more high level and human. Graphic design, not that different. API design replaces human with software, with slightly different capabilities, less volatile memory, more costly pattern recognition, and far more resiliency to boredom. Protocol design's consumer is the spirit of the convention, extensibility, and forward compatibility. Developer experience is User Experience in disguise.

Next step would be how to be the consumer. This is the easy part. The frontend consumes backend's interface via Web API and ajax, the process consumes machine resource via system calls, etc. Distributed instance consumes unity via RabbitMQ. It's the easier part because you're supposed to be served. Don't over-complicate.

I barely scratched the surface, but hopefully, this will set the right mindset. I exposed some keywords for you, the readers, to explore if you wish.

Parallel Programming: 1thread per core in Rust

Alan — Sat, 28 Mar 2020 04:51:23 +0000

This tweet boomed the other days. It's about the rare computation model of the newly released game by id Software, Doom Eternal.

The idea is foreign and crazy because when typically modern games would rely on data-oriented, CPU cache-friendly code first and having a single thread to handle render jobs, this game does differently.

The game creates a thread for each core to ensure all the CPU cores full. Data-oriented code is still critical toward performance, but the OP claims that keeping all CPU cores full is more important.

Axel Gneiting

@axelgneiting

Fun fact: Doom Eternal does not have a main or render thread. It's all jobs with one worker thread per core.

22:12 PM - 21 Mar 2020

309 2148

Architecture, either it's code, system, or information, always fascinate me, so does this novel computation model in the Doom Eternal. I'm also fascinated by Rust language and it's been months I'm learning more and more about it, though I haven't got the chance to write anything serious with it.

Tempted, I wrote a similar but simple computation model in Rust.

For the exercise, I wrote an executable that does simple CPU heavy calculations using a similar computation model. bcrypt hash is used as the CPU heavy calculation. The executable applies bcrypt-hash to 100 equal-length strings and writes the result into a file. I also wrote other approaches to compare with this model.

QThread

I love naming things, so I named this QThread.

QThread is a wrapper around a thread join handle and it's mpsc communication channel. The thread wrapped around has a specific job, receiving either pointers of functions and then executing them, or receiving None and then kill itself.

I just realized there's a cargo called QThread already that does entirely different thing, I should rename this someday

struct QThread {
    join_handle: JoinHandle<()>,
    active_arc: Arc<Option<()>>,
    tx: Sender<Option<Box<dyn Fn() -> () + Send>>>,
}

impl QThread {
    fn new() -> QThread {
        let (tx, rx) = channel::<Option<Box<dyn Fn() -> () + Send>>>();
        let join_handle: JoinHandle<()> = thread::spawn(move || loop {
            match rx.recv().unwrap() {
                Some(boxed_fn) => {
                    boxed_fn();
                }
                None => break,
            }
        });

        let active_arc: Arc<Option<()>> = Arc::new(None);
        QThread {
            tx,
            join_handle,
            active_arc,
        }
    }

}

To queue job into QThread, exec method is implemented into it.

The exec method receives a pointer of function. It wraps the pointer with another intermediate function, before passing it into the thread. This intermediate function creates a Weak, executes the inner function, and then drop it. The Weak is used to track the number of inner functions running.

The method active_count is introduced to get current running function by counting the number of Weak pointing to the active_arc.

impl QThread {
    fn exec(&self, func: Box<dyn Fn() -> () + Send>) {
        let strong = Arc::clone(&self.active_arc);
        self.tx
            .send(Some(Box::new(move || {
                let weak = Arc::downgrade(&strong);
                func();
                drop(weak);
            })))
            .unwrap()
    }

    fn active_count(&self) -> usize {
        Arc::weak_count(&self.active_arc)
    }
}

Remember that QThread will stop executing if it's sent None signal? schedule_kill method is introduced to schedule termination of the thread.

impl QThread {
    fn schedule_kill(&self) {
        self.tx.send(None);
    }
}

Now, we have a thread wrapper that receives a function pointer and executes the function. What's remaining is we will make multiple QThread and to achieve job balancing between multiple QThread, we need a regulator.

Let's make a QThreadRegulator. QThreadRegulator will contain many QThread.

struct QThreadsRegulator {
    workers: Vec<Rc<QThread>>,
}

impl QThreadsRegulator {
    fn new(worker_num: &usize) -> QThreadsRegulator {
        let mut workers: Vec<Rc<QThread>> = vec![];
        for i in 0..*worker_num {
            workers.push(Rc::new(QThread::new()));
        }
        QThreadsRegulator { workers }
    }
}

We need QThreadRegulator to help us find the currently available QThread. Let's make find_available_worker which returns the first QThread which has 0 active_count.

impl QThreadsRegulator {
    fn find_available_worker(&self) -> Option<&QThread> {
        for worker in &self.workers {
            if worker.active_count() == 0 {
                return Some(worker);
            }
        }
        None
    }
}

Last, we need some way to decommission all QThreads inside QThreadRegulator and to sync them all. We'll introduce schedule_kill and also wait_all.

impl QThreadsRegulator {

    fn schedule_kill(&self) {
        for worker in &self.workers {
            worker.schedule_kill();
        }
    }

    fn wait_all(&mut self) {
        let workers = &mut self.workers;
        while let Some(worker) = workers.pop() {
            match Rc::try_unwrap(worker) {
                Ok(worker_raw) => worker_raw.join_handle.join().unwrap(),
                Err(_) => eprintln!("join handle failed"),
            };
        }
    }
}

Using QThread

Now it's the turn to make the code that uses QThread to bcrypt-hash 100 equal-length strings. This code will be using bcrypt crate to bcrypt-hash the strings.

First, we open the input and output files. Then we create lines iterator for the input file.

pub fn qthreaded() -> io::Result<()> {
    let (in_path, out_path) = get_in_out_paths()?;
    let in_file = OpenOptions::new().read(true).open(in_path.as_os_str())?;
    let out_file_mutex = Arc::new(Mutex::new(
        OpenOptions::new()
            .read(true)
            .write(true)
            .create(true)
            .open(out_path.as_os_str())?,
    ));
    let mut lines_iter = BufReader::new(&in_file).lines();

Create the QThreadsRegulator.

    let mut worker_regulator = QThreadsRegulator::new(&num_cpus::get());

Loop the lines iterator. For each line, there's a nested loop inside. In the inner loop, QThread's find_available_worker will be called to see if there is any available QThread. If there's one, pass a job function pointer into it and move on to the next line. If there's none, simply sleep to let the main orchestration thread rest.

    let mut iteration = 0;
    for line in lines_iter {
        let pass = line?;
        println!("qthreaded count: {}", iteration);
        iteration += 1;
        if pass.len() > 0 {
            loop {
                let pass_clone = pass.clone();
                let out_file_mutex_clone = Arc::clone(&out_file_mutex);
                match worker_regulator.find_available_worker() {
                    Some(worker) => {
                        println!("worker-found");
                        worker.exec(Box::new(move || {
                            let pass_u8 = pass_clone.as_bytes();
                            match hash_and_write(&pass_u8, &out_file_mutex_clone) {
                                Ok(_) => (),
                                Err(_) => (),
                            }
                        }));
                        break;
                    },
                    None => {
                        println!("no-worker-found");
                        std::thread::sleep(std::time::Duration::from_millis(5));
                    },
                }
            }
        }
    }

Finally, kill the QThreads and wait for all QThreads to finish their jobs.

    worker_regulator.schedule_kill();
    worker_regulator.wait_all();

    Ok(())
}

We're done.

The Repo

Here's the repository where you can clone and cargo run it.

Kelerchian / expr-bcrypt-qthreaded

One thread per core in Rust

expr-bcrypt-qthreaded

Intrigued by Axel Gneiting's twitter post about Doom Eternal jobified system I created a snippet of how we can do it in Rust.

More thorougly explained in this dev.to post of mine

View on GitHub

Inside the repo there's also 2 more calculations method to be compared, both are doing the same, bcrypt-hash-ing the same 100 equal-length string with the same bcrypt cost.

The first is sequential, sequential is simply one thread running the bcrypt-hash. I created it to see how much faster QThread is compared to doing it with a single core.

The second is batched. Batched is similar to sequential, but each iteration will create threads as many as available CPUs in the device, and run concurrent bcrypt-hash. It's not so much a fair comparison to the QThread one because in this one the out_file are written using the main thread.

Disclaimer:
QThread and QThreadRegulator are runnable but not perfect. There are many things to fix, few of them are:

No recovery from panic in the thread.
No job graph like Doom Eternal.
Weak calculation is inaccurate due to it being created after the intermediate function is run, not before.
Decommissioning QThread should use impl Drop.
Dynamic QThread number inside the regulator (imagine plug-and-play processor in the future!)

It's a POC for fun and educational, but I might make it a full-fledged crate in the future.

Comparing time

The time needed for QThread method is the fastest of the three. Batched and QThread has almost the same time though, while sequential need 4 times the time required by batched and QThread.

As of why QThread is always faster is maybe because either 1.) there's a cost in creating a thread or 2.) the one thread file write is slow.

Results will vary between machines.

Conclusion

Not much to conclude here, the comparison I made is not exactly fair and there's not much metric to see other than time, there's so much more to explore like core usage.

From this experiment, I find that writing 1thread/core code is not as complex as I thought it would be. I can still read the code easily despite the new necessary complexity. There's an additional need to check for an available thread but that's it.

This computation model may be useful for other fields of computing. Now, while it is now a hidden treasure laying in the Doom Eternal codebase, may this blog post and the tweet sparks interest to the computation model to other games and other fields of computing. Unfortunately, multi-threading is now unavailable in wasm and that's a part where I would really like to explore, making a games in wasm and HTML Canvas.

Rust is really fascinating. It is surprisingly easy to write in it despite it is as low as c++ in terms of its closeness to bare-metal. And that makes me continue to love Rust. It's been fun, and moreover it is actually easy to write for people used to operate on a higher-level language. I write TypeScript every day and I don't feel any difficulties moving paradigm from TypeScript to Rust.

That's all, thanks for reading.

Concept: Cohesive Non-Coupled Frontend Code Architecture

Alan — Sat, 07 Dec 2019 05:08:47 +0000

In this article, I'm going into the more detailed part of the software development, code architecture.

I have been working on a TypeScript front-end app for browser and electron in the fast-paced gaming industry for 2 years surrounded by brilliant colleagues. It has its perks. It inspired me to develop a concept for cohesive, non-coupled code architecture, for front-end client-side application. Here it is.

The architecture

The architecture. Keep it in a new tab for easy access while reading this article.

It's only fair if "the why" is explained before "the how". Feel free to skip to "the how" if you need to.

The Why

Low coupling, high cohesion. The phrase pops up on software development articles and books not without reason.

To change a feature in a program, first, you need to absorb the knowledge in the said code, like "what it does?", "why is it written that way?", "is there any implicit behavior not apparent at first sight?". Identifying those allows developers to accurately make changes to a feature, reducing the chance of regression from the changes' side effects. The more varying the knowledge put closely together, the harder it is to understand, the later the delivery.

Multiplies that with the number of engineers involved in a project. This is why for every N engineer that is involved in a project, only (log N - 1)-ish velocity is being added into the team.

Minimizing this effect can be done by accurately grouping and separating codes so that the more codes relate, the closer they are organized. This is high cohesion. At the same time, codes that don't relate should be separated and should not depend on each other. This is low coupling. Emphasis on the word accurately, if one manages to modularize codes in a manner where it becomes harder to understand, the effort is in vain.

The fact that there are multiple dimensions of relations between codes. It can be how the pattern they are written, the business logic they are handling, the role it is taking, the layer it is placed on, etc. The challenge is to strategically place the codes so that they are predictable, readable, usable, and expendable at the same time. Developers should be able to plug in codes quickly without worrying about side effects and regressions and they should also be able to plug out codes, remove lines, without concurring much damage in the form of regressions.

The How

Look at the image. To the left is how the code should be organized, and to the right is how the objects relate to each other logically on runtime.

Codes should be separated by 1.) scale (e.g. global and local, global, local), 2.) layer placement (e.g. network call layer, logic layer, presentational layer, utilities, helpers), and then 3.) knowledge (e.g. account management, task management, about page, welcome page, etc). These orders are only guides and are not absolute rules, most probably but not always the best practice.

The left: how modules should be organized.

In the image above, codes are grouped by layers. Three kinds of modules are involved: 1.) API calls, 2.) Business Logic, 3.) Presentation Layer. Feel free to add more layers to your app to your liking (e.g. i18n layer, storage adapter layer, codec layer, etc).

There are some constraints that must be enforced on each module:

API Calls modules should only concern translating fetch/WS calls into business objects.
Business Logic modules should include data structure, state lifecycle, and actions/state transformer concerning only business logic
Presentation layer modules should only concern presenting data provided by business logic modules and additional UI-only features.

Cross-function data access and method calls should be minimized between layers. This means no react component, DOM modification components on business logic and API calls, no fetch() on presentation layers. This is to minimize couplings.

In the image above you can also see EventEmitter. Imagine EventEmitter as an entity that can be subscribed based on eventName, "downloadSuccess" for example. Other entities can also trigger the emission of "downloadSuccess" together with a payload into that EventEmitter which will trigger subscription functions previously registered by subscribers. EventEmitter should also have type safety definition, meaning that each eventName should have a type definition for its payload. For example "downloadSuccess" would have a payload of string which indicates the path of the file which had succeeded the download. This allows infinite-way communication between entities that have reference to it.

/
/src

/src/api                # global api calls
/src/api/session.ts
/src/api/users.ts
/src/api/account.ts

/src/app-state                # global module
/src/app-state/index.ts       # the application-state business logic
/src/app-state/session.ts     # member of application-state
/src/app-state/locale.ts      # member of application-state
/src/app-state/chat.ts        # member of application-state

/src/modules
/src/modules/dsm-region                       # local module
/src/modules/dsm-region/api/dsm-region.ts     # local api calls
/src/modules/dsm-region/models/dsm-region.ts  # local business logic
/src/modules/dsm-region/dsm-region-logic.ts   # local business logic
/src/modules/dsm-region/dsm-region.tsx        # local presentation layer
/src/modules/dsm-region/dsm-region.scss       # local presentation layer
/src/modules/account                          # local module
/src/modules/account/account-logic.ts         # local business logic
/src/modules/account/account.tsx              # local presentation layer
/src/modules/account/account.scss             # local presentation layer

/src/index.tsx     # global presentational layer
/src/index.scss    # global presentational layer

Example of modules placement

Suppose a requirement came, one which says "change a select element in the account management page into radio buttons". You're full of other equally important works, and you want to delegate it to a newer developer joining the team a few days ago.

No worries, they'll edit the presentation layer and expect no side effects on the business logic. :D

By now, you might be tempted to group together similarly looking codes to push up the cohesion a.k.a. The Abstraction/The Don't Repeat Yourself. To abstract or not to abstract? Layer separation should be on your mind before abstracting things out of it. Low coupling is more crucial than high cohesion. This order of separation avoids common modules that are annoying but you don't know why. You're not going to need it.

This has been all theory, no real code involved. So here it is.

THE HOW

I'll use these tech stacks to show how the architecture in action:

Presentational Layer: React
Business Logic Layer: Unstated, built-in event module
API Calls Layer: Fetch API
Structure Definition: io-ts

// /src/modules/dsm-region/models/dsm-region.ts

import * as t from "io-ts"

export const DedicatedServerManagerCodec = t.type({
 type: t.keyof({
  "stable": null,
  "beta": null
 }),
 id: t.string
});
export type DedicatedServerManager = t.TypeOf<typeof DedicatedServerManagerCodec>;

// The type definition above is the equivalent of 
// export type DedicatedServerManager = {
//   type: "stable" | "beta",
//   id: string,   
// }

Above is the code of the definition of DedicatedServerManager. It is an io-ts codec so that it can serve two purposes: type definition and type guard.

// /src/modules/dsm-region/api/dsm-region.ts

import * as t from "io-ts"
import { apiBaseUrl } from "/src/config"
import { DedicatedServerManagerCodec, DedicatedServerManager } from "../models/dsm-region"

export const fetchAvailableDSM = async (): Promise<{ value: DedicatedServerManager[] } | { error: E }> => {
  const response = await fetch(new URL("regions/dsms", apiBaseUrl).toString())
    .catch(error => ({ error }))

  if (response.status < 200 || response.status > 399){
    return { error: new APIError() }
  }

  return response.json()
    .catch(error => ({ error: new DecodeError() }))
    .then((json) => {
      if(!t.array(DedicatedServerManagerCodec).is(json)) {
        return { error: new DecodeError() }
      }
      return { value: json }
    })
}

Above is the network layer of the DSM module. As a network layer module, its only concerns are to get and send data via the network. Also, to transfer data correctly it needs to parse it using type guards defined in "../models/dsm-region". It is assured that the consumer of these functions will always either receive the correct type of the data either on the run-time or receive an Error object explicitly, never through exception, which type validation is never properly supported by TypeScript

// /src/modules/dsm-region/dsm-region-logic.ts

import { Container } from "unstated"
import { DedicatedServerManager } from "./models/dsm-region"
import { fetchAvailableDSM } from "./api/dsm-region"

type DSMAvailabilityMap = Map<"stable" | "beta", DedicatedServerManager[]>;

export class DSMRegionPageLogic extends Container<{
  isFetching: boolean
  dsmMap: null | DSMAvailabilityMap
}>{

  events: EventsEmitter<{
    fetch: void,
    fetchSuccess: void,
    fetchError: Error
  }> = new EventsEmitter();

  state = {
    isFetching: false,
    dsmMap: null
  }

  async init(){
    try {
      if(this.state.isFetching) return;
      this.setState({ isFetching: true, dsmMap: null });
      this.events.emit("fetch");

      const availableDSMs = await fetchAvailableDSM().then(result => {
        if(result.error) throw result.error
        return result.value
      });

      const dsmMap = (["stable", "beta"]).reduce((dsmMap: DSMAvailabilityMap, dsmType) => {
        dsmMap.set(dsmType, availableDSMs.filter(dsm => dsm.type === dsmType));
        return dsmMap;
      }, new Map());

      await this.setState({ dsmMap })
      this.events.emit("fetchSuccess");
    } catch(error) {
      this.events.emit("fetchError", error);
    } finally {
      this.setState({ isFetching: false })
    }
  }

}

Above is the logic part of the DSM module. The logic part of the DSM module is a very simple one. It has two states, isFetching and dsmMap. It has one method, which is to fetch the data while at the same time track the fetch process through the isFetching state.

This logic module has a special EventEmitter object events composited in it. It provides a mean of communication between the logic module and its consumer. This logic module broadcasts its events through the events object to tell the consumer what is happening inside.

// /src/modules/dsm-region/dsm-region.tsx

import * as React from "react";
import { DSMRegionPageLogic } from "./dsm-region-logic"
import { DedicatedServerManager } from "./models/dsm-region"
import ErrorBanner from "src/components/common/ErrorBanner";
import LoadingSpinner from "src/components/common/LoadingSpinner";
import styles from "./dsm-region.scss"

type Props {}
type State {
  error: Error | null
}

export default class extends React.Component<Props, State> {
  logic = new DSMRegionPageLogic();
  state:State = {
    error: null
  };

  componentDidMount(){
    // subscribe is unstated's method to listen to state change
    this.logic.subscribe(() => this.setState({}));

    this.logic.events.subscribe("fetch", () => {
      this.setState({ error: null })
    })
    this.logic.events.subscribe("fetchError", (error) => {
      this.setState({ error });
    })
  }

  render(){
    const { error } = this.state;
    const { dsmMap, isFetching } = this.logic.state
    return (
      <div className={styles.dsmRegionPage}>
        { error && <ErrorBanner error={error}/> }
        { isFetching && <LoadingSpinner text={"Please wait. Loading DSM data."}/> }
        { dsmMap && (
          <div className={styles.dsmSections}>
            <DSMSection dsms={dsmMap.get("stable") || null} />
            <DSMSection dsms={dsmMap.get("beta") || null} />
          </div>
        ) }
      </div>
    )
  }
}

const DSMSection = ({ dsms }: { dsms: DedicatedServerManager[] | null }) => {
  if (dsms == null) return null;
  if (dsms.length === 0) return null;
  return (
    <div className={styles.dsmsSection}>
      {dsms.map(dsm => (
        <div className={dsm}>
          <a href={`/dedicated-server-managers/${dsm.id}`} >{dsm.id}</a>
        </div>
      ))}
    </div>
  )
}

Above is a page component using DSMRegionPageLogic. It has a very small number of states and a very simple lifecycle thanks to the separation. This page component is allowed to concern only about managing UI states and not care about everything else (network, data/logic lifecycle). The only concern it has is that if the logic object emits an error, which it has to catch and show.

Being decoupled, changes to these components can be easily made. UI changes will not affect logic, Logic changes will not affect the network layer, and so on. It cannot be said that is the same the other way because of the dependency arrow but it helps a lot with that.

These codes above are only a small, simple example of the decoupling. This decoupling that leverages separate lifecycles, state scoping, and event system could be utilized much more. Communication between components from separate VDOM tree becomes possible without redux/flux-like god-object.

Clean Dependency Graph and Identity of a Code

Alan — Fri, 28 Jun 2019 17:20:17 +0000

Clean dependency graph and Identity of a code
Next: Ending the decades of war between declarative and imperative code - Complexity minimization is a form of tail risk management - Functional paradigm brought up to component level

Satisfying my newfound obsession of pouring my thoughts, this time not in the form of code, I'm sharing one of four random things I realized while rewriting a large, crufty code base.

This article is about clean dependency graph and identity of a code.

I'm using typescript, TSX (the typescript counterpart of the-now-popular-in-the-web-community JSX), and a little dose of React in this article. I hope you can see it the same as any other codes, even read it like a story.

Clean Dependency Graph

A dependency graph

The work that I'm doing is pretty rare in nature. One of the challenges we need to solve is to write a conveniently forkable code that is easy for git merging and is customizable - an intimidating combination.

Codes shown below are example codes analogous to the real one. Due to the real one being proprietary, I cannot show any of it (not that I want to).

Clean dependency graph yields flexible codes. A dependency graph is a directed graph representing dependencies between modules in your code. For instance, A renderer module A importing a class of shareable state B would be A->B.

A golden rule for a clean dependency graph is if it is roughly divided horizontally and vertically, it fulfills these premises:

Modules in the same row have similar methods
Modules in the same column have the same domain
Modules are vertically sorted from the least volatile to the most volatile and arrows should never point down.

Modules in the same row should have similar methods

Modules in the same row has similar role for different domain.

Take the example of a group of API Calls modules:

// UserAPI.ts

export function fetchUserById({ userId }:{userId:string}){
  return decodeOrError(
    () => networkManager.withCredentials().get(`http://domain/some/path?userId=${userId}`),
    UserDecoder,
    UserDecoderError
  )
}
// DepotsAPI.ts

export function fetchBuildByBuildIds({ buildIds }:{buildIds: string[]}){
  return decodeOrError(
    () => networkManager
      .withCredentials()
      .get(`http://domain/api/builds/?buildIds=${buildIds.join(",")}`),
    BuildDecoder,
    BuildDecoderError
  )
}

// TagsAPI.ts
export function addTag({ tags },{tags: string[]}){
  return decodeOrError(
    () => networkManager
      .withCredentials()
      .post(`http://domain/api/builds/?buildIds=${buildIds.join(",")}`),
    Decoder.unknown,
    CommonDecodeError
  )
}

Three modules concerning three different domains are done alike and form a role for the modules, which in this case is to call API endpoint, decode, and guard and cast type. Development wise it's easy to copy and paste codes between modules with the same roles and their tests, thus reducing cognitive load. It applies to either a team of developers or a single developer.

Modules in the same column have the same domain

This is pretty straight forward and intuitive.

The concept is similar to micro front-end where separation of concern is the key. It creates a clear definition of business logic, right from the model definition, to the presentation layer.

// BuildModel

export const BuildDecoder = createDecoder({ 
...
})
export type Build = Type<typeof Build>

// BuildAPICall

import { BuildDecoder } from "src/module/build/model"

export function fetchBuilds(){
  return decodeOrError(
    () => networkManager
      .withCredentials()
      .get(`http://domain/api/builds/`),
    Decoder.array(BuildDecoder),
    BuildDecoderError
  )
}

export function addBuild({ build }: Build){
  return decodeorError(
    () => networkManager
      .withCredentials()
      .post('http://domain/api/builds/', build),
    BuildDecoder,
    BuildDecoderError
  )
}

// BuildsViewState

import { StateManager } from "src/utils/state-manager"
import { Build } from "src/module/build/model"
import { fetchBuilds, addBuild } from "src/module/build/api"

type State = {isFetching: boolean, builds: Build[] | null, error: Error | null}

export class BuildsViewState extends StateManager<State>{
  state: State = {
    isFetching: boolean,
    builds: null,
    error: null
  }

  // override empty init method
  init(){
    try{
      if(this.state.isFetching) return
      this.setState({ isFetching: true })
      this.setState({ builds: await fetchBuilds(result => {
          if(result.error) throw result.error
          return result.response.data
        }) 
      })
    } catch(error) {
      this.setState({ error })
    } finally {
      this.setState({ isFetching: false })
    }
  }

  // inherits empty deinit method
}

// BuildViewPage

import { React } from "react"
import { Loading, CommonError } from "src/common/components/loading"
import { BuildViewState } from "src/utils/build/page/view"

export class BuildViewPage extends React.Component {
  sharedState: new BuildViewState();

  componentDidMount(){
    this.sharedState.init()
    this.sharedState.subscribe(() => this.setState({}))
  }

  componentWillUnmount(){
    this.sharedState.deinit()
  }

  render(){
    const { isFetching, builds, error } = this.sharedState.state
    return (
      <section>
        {isFetching && (
          <Loading>
            Loading your Builds. Please Wait.
          </Loading>
        )}
        {error && (
          <CommonError error={error} />
        )}
        {builds && builds.map(build => (
          <pre key={build.id}>
            {JSON,stringify(build, null, 2)}
          </pre>
        )}
      </section>
    )
  }
}

An intersection between the row and the column creates an identity of the module, say the app is a web app to manage builds of software versions and it has a BuildViewPage - BuildViewPage can be defined as a module which present view (role) of the build (domain).

The key is in the next rule.

Modules are vertically sorted from the least volatile to the most volatile and arrows should never point down.

These days, importing other modules is as easy as pressing alt+enter, and even some IDE support not-pressing-anything feature to do that. Let's call it import convenience. Import convenience pulls us away from contemplating on why and how do we import modules, or in my word, the art of keeping dependency graph clean.

This rule siphons the essence of importing, that less volatile module should not import more volatile module. Volatile here refers to being prone to changes.

Violation of Rule 3. A module imports from a more volatile module

Sometimes it is tempting to import anything without thinking of its consequences, it's never forbidden by compilers anyway. But notice that when a dependency module change, the dependent module may also change. The dependent may change explicitly (need a change of code), or implicitly (changes are inherited).

Let the fact below be true:

let B->A or A<-B means B depends on A

AND:
- A<-B
- B<-C
- B<-D
- B<-E

When A changes B, C, D, and E may also change.
A change in module A results in, at least 0 changes, at most 4 changes.
Which means at least 0 additional effort for a code change, at most 4 additional efforts for the code change. And an exact 5 additional efforts to test the changes.

So at least 2 unit of work, at most 10.

I might sound lazy for calculating a very little amount of changes, until when this rule is applied on a much grander scale, on a big project with a complicated dependency graph.

But it was only a small reason why I needed the rewrite. The reason was that the old code didn't have clear visibility of the graph.

"Rule 3 doesn't make sense"

In first glance, Rule 3 sometimes doesn't make sense.

For example, here is a file that stores information about Build. This file is imported everywhere, including helper modules, factory modules, presentational modules, shared state modules, etc.

// build/model.ts

type Build = {
  id: string,
  version: string,
  patches: Patch[]
}

type Patch = {
  srcUrl: string,
  position: number
}

type BuildResponse = {
  data: null,
  errorCode: number
} | { 
  data: Build[], 
  errorCode: null 
}

function deduceBuildError(errorCode){
  switch(errorCode){
    case 1000: return "Build not found"
    case 1001: return "Build corrupt"
    case 1002: return "Build being switched"
    default: return "Unknown Error"
  }
}

The twist is that deduceBuildError is used by the presentational module to render error message and there is a lot of requests for changes to the deduceBuildError function for UX reason. Although the change should only implicitly affect presentational module, it risks of other module being implicitly changed.

This is due to the corrupted Identity of the file build/model.ts. Being a model it should not have deduceBuildError which deduce error message based on the errorCode. Deducing build error message is simply not its role.

The correct way is to move it out of the model. It can be inside the presentational module since it is the only thing supposed to be affected by its change. The Identity of the presentational module is not corrupted because the role of the presentational module after the addition of deduceBuildError doesn't change. It can also be put into another file that resides right above presentational module.

The dirtiest kind of dependency graph

The dirtiest, worst kind of dependency graph, is to not have it and to have no guard from having a nonsensical kind of dependency, the circular dependency.

Having unclear or no dependency graph would mean:
No clue on where to add things
No clue on when to change things
No clear decision on when to apply DRY or to copy-paste
No clue on when to pull things out of modules
No rapid development
No productivity

Clean Dependency Graph in System Architecture Level

Dependencies happen between system architecture level. The likeliness of it being dirty is a lot less than codes as it moves slower than the change of code. Nevertheless, a misstep would cause problems that often happen to inter-component interactions, for example, bottlenecks on network calls.

Identity of a component is what you must focus on. Putting a feature not in a proper place, misnaming an endpoint, implicit behaviors, would make the system leans to a more dirty dependency graph.