Projects in 2017 - 2023

Introduction

This article introduces my experience in 2017 - 2023. Now I quit my local country's company and came to Vancouver, Canada. Originally, I wrote this article for my college's task and I think that was a good opportunity to review my experience. However, it got too long to read for that purpose, so I decided to divide and post this independently. Only the parts that have permission to publish by ex-company are described.

Projects Index and Overview

I am writing this in 2023, and I have worked for 6 years on software-related works for now. I'll introduce 3 projects in this article that I'm passionate about. Besides these, there are other projects that I have experienced.
In a department that I belonged to, there was only one developer that is me. So, I worked from planning to maintenance and instructed other developers.

• 1.1. Visual Inspection Machine

To help workers work and to generalize inspection work, create a visual inspection machine.

• 1.2. Core Business System

This project started in 2020 the year of the COVID-19 pandemic. We had to change our work style from Onsite to Remote (Hybrid) without any cost. So, I created this system. I set the priority to speed rather than quality. Then I realized this system was (going to) the technical debit. Therefore, I had to redesign and refactor this system after this system was released.

• 1.3. IoT System

Data aggregation system. This system aggregates data on equipment or the environment of the factory, then visualizes and analyze it to prevent equipment defect. Finally, This system was merged into a company-wide system.

1.1. Visual Inspection Machine

• Problem

In the factory where I worked, there are visual inspection process, and workers used a loupe to check parts and write down in parts detail on paper. This process seems difficult in addition this process is a critical process. But there were only two workers, a total worker of this team is five people, who could do this process at that time. Therefore this process sometimes could become a bottleneck.

• Solution

I created a visual inspection machine. It is a fundamentally XY table that holds a web camera. I created it by modifying this kind of product. I chose the Raspberry Pi as the PC that controls the table because I need a digital I/O pin to communicate with the driver board of the table. We can know the coordination of parts and it can easily convert to a .csv file, so I created a GUI and a table control program using Python. Users (workers) just need to set a work and click a button. Then, the camera moves to target parts and the display of the PC (RPi) shows the parts with expanding

Using this machine, every member of the team can now do this work. And working time also reduced.

• Skills

Python

1.2. Core Business System

• Problem

In 2020, the COVID-19 pandemic covered the world, and we had to change the work style from Onsite to Remote (Hybrid). However, the department's system was created by MS Excel and MS Access. So, to share the data, a person who went to work had to upload that data to cloud storage and, persons who work from home downloaded it from a cloud. Some of those data sizes were over 1 GB, so downloading took sometimes 1 hour. At the same time, spending was curtailed so we couldn't prepare enough budget.
Creating a product needs several processes. maintenance master data, process-designing, publishing designed processes to workers, keeping a record of work (who and when did), and so on. These days, these are managed by some strongly bounded independent systems. To publish data to workers, papers were used which means someone has to print, write to it, scan, and upload data to some server(s).

• Solution

We had to change this situation as soon as possible. It could handle this situation by hand but it was very time-consuming. To handle this situation, I reviewed the business process and created a new core business system using AWS.
The system is a serverless web system. In the first phase, it was quite simple, the backend was the web three-layer application and just handled CRUD using Python, and the frontend was React with Typescript. The interface between frontend and backend is GraphQL, but since this system is also used by legacy systems or tools, this system exposes read-only REST API. After I created this system, it seemed to work very well. The spending time related to is reduced 1000 hours/year and 1 person who works for ex-system.
And I noticed this system was very hard to maintain.

You already know the process of work is not stable. Some problems will come out, the restriction will change, and external environments force to change process. This application had just rolled out so there are many feedback from users. In addition, new requests came for new features that integrate with other systems.
As I mentioned above, I had to create a new system as soon as possible so my priority was speed rather than quality. But I realized that this was the wrong decision. It was hard to understand the overall picture, it was had poor models, and it hard to check influence when trying to make changes because I used Python as a backend language which has weak typing. Therefore I had asked permission to redesign and started creating version 2.
I adapted TypeScript as a backend language because it has a strong typing system and is already used to frontend and IaC (CDK). And I decided to adopt Domain Driven Development. Fortunately, I was familiar with the domain of my department's work, we could easily reach an agreement about definitions of each word (ubiquitous language). In addition, there was a plan that scale this system that developing a production planning system asking for outsourcing. Thus, I divided this system, into a microservice, and adapted an event-driven architecture to publish events to other systems.

• Skills

Version 1: Python, React.js, TypeScript, AWS(Lambda, CDK, AppSync, API Gateway, Aurora Serverless)
Version 2: TypeScript, React.js, AWS(Lambda, CDK, AppSync, API Gateway, DynamoDB, Aurora Serverless, Event Bridge)

1.3. IoT System

• Problem

For factory productivity, it is important to prevent machine defects or to gather data to determine the cause of some production defective. Actually, this project started as an experimental project. There was a few problems like some data was written down to papers and hard to access to it. But in most cases, there were few chances to need that kind of thing. Although 2018 this project started, it was also a fact that some factories get profit by preventing machine defects or detecting some problems not only with machines but also with products by correcting and analyzing data from equipment.

• Solution

I created some small systems that were designed for a single kind of equipment and sensor. I think it worked well especially, with a checking mechanism we could know some bad conditions before it gets to the defect. Using this, we could reduce the cost and time spent on recovering defects.
Some years later after deploying those systems, similar needs emerged in other equipment. However, those systems were designed for each equipment thus I need to design a new system that will be more general.
Before explaining a new system, I would like to explain the data in this context. Generally, the equipment in the machine industry is controlled using PLC or the same kinds of devices. The equipment data is mainly generated by sensors and the PLC aggregates those data for controlling equipment. For example, if there was an equipment having a furnace. It may have temperature sensors and heaters and PLC gets temperature data and controls heaters if exceeds the temperature limit, heaters turn off. It means the PLC has numerical data.
The previous one was deployed on a server that I managed, but it was hard to manage it. The previous one has access to the PLC to extract data. To reduce management costs, the new system uses AWS and changes the data-extracting strategy. I created a server program that receives data from equipment and converts it to HTTP then send data with credential to AWS IoT Core. This program accepts UDP requests from equipment and extracts data from its datagram. Generally, the PLC cannot create HTTP requests or even if it can, the using it are very restricted and cannot modify but it has an L-4 connection TCP or UDP. Therefore I created a server program that accepts UDP requests from equipment. The frequency of requests from equipment is predicted to be less than 10 msec in the worst case, and this program will run on the shared PC in my department. What's more, the number of equipment was not predictable. Hence I had to create a light wait and avoid using a gavage collection so I used Rust for this server program.

The previous one accessed the PLC or sensor. This means when the change occurs (like when new equipment is installed, the new sensor is attached to equipment), the system must be changed. But there is a lot of equipment there and each equipment is assigned person in charge. So the target of the element (i.e. temperature, air pressure, and so on) must be decided by those people. Because those people have deep knowledge about equipment or problems with equipment but they don't have software knowledge. So, to change occur, the person in charge must ask me to update the program. It robs us of productivity and speed. So new one takes a strategy that just accepts data from equipment. Of course, it needs to change the settings of the program. But it is outsourced, so the person in charge can change settings themselves then the settings are reflected in the program.

This system works well and the same needs exist in other factories in the ex-company, as a result, this system is integrated into a company-wide system.

• Skills

Previous one: Python, Django, TypeScript, React.js, MariaDB
New one: Rust, TypeScript, AWS(IoT Core, Kinesis Firehose, S3, CDK, API Gateway, DynamoDB, Aurora Serverless, Event Bridge)

Conclusion

This was a good opportunity to review what I did. I learned many things from these experiences. When I start a new project, I will remember these experiences to reduce unnecessary nightmare maintenance.