DEV Community: makotunes

Best Practices for Engineer Evaluation Systems in the Age of AI (Overview)

makotunes — Mon, 21 Jul 2025 14:06:48 +0000

Introduction

I run a development company with around 40 people. In recent years, AI-driven development has become the norm, and the acceleration brought by the era of vibe coding—together with the growing difficulty of fairly evaluating engineers—motivated me to develop an engineer evaluation platform: CodeRanker.

Source Code

⸻

Background

External Factor: AI

With the rise of so-called “vibe coding” (real-time code generation and thread-based workflows powered by AI), there’s been a clear shift from traditional team-based division of labor to a relative emphasis on individual ability. It’s not that engineering jobs are disappearing—rather, it’s akin to the emergence of a new high-level language and a fundamental change in the cognitive demands required of engineers.

For clarity, the “AI” here refers to vibe-coding-style AI support tools like Cursor. However, I see this trend as a transitional phase. In the near future, development will converge further toward declarative, structured, spec-driven paradigms (for example, Kiro), where prompts themselves become declarative code. Even as technology advances, the fundamental trend toward the abstraction of thought—where development is completed by specifying logic and requirements—will remain unchanged.

The reason I believe we will shift from vibe coding to declarative, spec-driven development is the expected increase in LLM model context window sizes. If cheap, widely available models can handle context windows of several megabytes—the size of an average codebase—even prompt-level adjustments may no longer be necessary.

As a result, future engineers will need to possess abstract thinking skills: the ability to see the entire system, maintain coherence, and define requirements across modules, rather than simply breaking work into modules and dividing tasks.

This means that, in the development environment of tomorrow, having one person manage the whole—leveraging AI—will naturally result in higher quality output, rather than splitting up work among many people.

Division of Labor Will Be Disrupted in the AI Era

Therefore, I no longer see value in teams composed solely of simple coders. Ideally, projects should be handled by a small group—or even a single person—who can see the whole picture. Just as society shifted from agriculture to hunting, traditional SIer-style, large-scale waterfall development will be eliminated. I believe that engineers will be regarded more as talents than as organizational “employees.” In this world, the difference in productivity by individual ability can be tenfold or a hundredfold. Engineering and other intellectual work should be the domain of a select few, without the unnecessary overhead of communication costs.

If a product is built by one person, its value is equivalent to the value of that developer. Thus, I consider this field to be more like the service or sales professions—where results-based incentives and a supporting evaluation system become necessary.

How Should Humans and AI Coexist?

There were also internal factors in my organization: with a lack of senior engineers, we hit the limitations of conventional management, training, and evaluation systems. The challenge became: how do we boost the performance of a team made up mostly of mid-level and junior engineers? The answer, I believe, is to use AI for correction and upskilling, but also to help engineers recognize the areas where AI falls short and develop human abilities that surpass AI—ultimately ensuring client satisfaction.

AI as the “Tool,” Humans as the “Judges”

AI can help with quality assurance and coverage measurement, but if AI were perfect, there would be no need for humans. In reality, AI alone cannot yet determine the quality or requirement-fit of complex systems. The meaning of requirements, technical depth, and the thoroughness of testing still demand the insight of senior engineers. AI should play a supportive role. To compensate for the thin senior layer, it’s important to combine automated measurement by AI with human judgment, and to continue on-site learning and review.

Treating Engineers as “Talents,” Not Just Organizational Members

Traditionally, organizations have often treated engineers as just parts of the company, making promotions and compensation decisions based on a manager’s evaluation. In my organization, we value engineers as talents based on three axes: quality of output, volume of output, and client satisfaction. This ensures fair evaluation according to each engineer’s projects and skill set, and makes their roles and rewards objective.

360-Degree Evaluation and a Democratic Process

To guarantee transparency and fairness, we use 360-degree evaluation by multiple reviewers. Deliverable evaluation, commit evaluation, and manager evaluation are all made public. Evaluation standards and calculation methods should be clearly documented and disclosed. Transparent and democratic evaluation is the basis for a flat, decentralized organization—not a top-down one.

Transparency and Decentralization

Another important factor for organizational growth is transparency. We focus on publishing rankings, instant reflection of evaluation results, and sharing of criteria, so all employees act with the same information. Flat evaluations—where hierarchy is minimized—reduce central authority and foster self-management and autonomy.

Proactive Competition

Fair competition improves productivity across the organization. We make monthly rankings public and link them directly to rewards and project assignments. This encourages ambition and covers for a thin senior layer by motivating everyone to improve. When connecting rankings and rewards, it’s important to clarify evaluation criteria and enforce penalties for misconduct, maintaining a healthy competitive culture.

Toward a Self-Regenerating Organization

The ideal is an organization that can grow and renew itself with minimal management cost. By combining quantitative AI-based evaluation with qualitative human judgment, and designing a transparent, decentralized evaluation process, each engineer understands their own strengths and challenges and takes initiative for improvement. This balance of competition and co-creation is the future of talent management and evaluation for the AI era.

This article outlines the concepts and practical framework of the CodeRanker engineer evaluation system as an organizational theory.

What is the 3-Axis Evaluation System?

The HR evaluation system proposed by CodeRanker measures an engineer’s performance on three axes: Quality, Quantity, and Client Evaluation.

Axis	Content	Purpose
Quality Evaluation	Senior engineers (with AI support) evaluate deliverables: code quality, design, test coverage, technical depth.	Guaranteeing and standardizing technical quality
Quantity Evaluation	Analyzes Git commit history. AI evaluates the quantity and quality of output by parsing commit messages and file changes.	Objective visualization of output volume
Client Evaluation	Project managers or clients assess requirement fit and business value.	Reflection of business value and customer satisfaction

By combining these, a balanced evaluation is possible, integrating technical, quantitative, and customer perspectives.

The Structure and Theory Behind the 3-Axis Evaluation System

Modern software organizations need mechanisms to correctly and fairly assess engineering ability. CodeRanker was designed to meet this need, with the 3-axis evaluation system as its core. This section explains the framework and underlying theory behind the system.

Why a New Evaluation System is Needed

Advances in AI have transformed engineering. With automated code generation and sophisticated CI/CD, individual productivity has increased dramatically. As a result, it’s become difficult to judge “who creates real value” using only traditional, subjective, or qualitative methods. Seniority- and impression-based evaluation makes it hard to recognize truly outstanding talent and risks harming organizational competitiveness.

This is why a data-driven, highly transparent evaluation system is needed. By visualizing performance with clear, quantitative data, you can fairly assess each engineer’s effort and results, and boost motivation. CodeRanker’s 3-axis system is designed to objectively measure “true value” in the AI era.

Three-Dimensional Evaluation

The 3-axis system assesses engineers from the following perspectives, with each axis addressing a different aspect to achieve an unbiased, comprehensive evaluation:

Deliverable Quality (Output-Focused): Evaluates the quality of code and products produced by engineers. Assesses requirements coverage, test completeness, design quality, security, and performance. Evaluation is based on automated test results and code reviews by senior engineers using specialized tools. This axis focuses on the quality of results—emphasizing technical depth and code completeness.

Quantity of Output (Process-Focused): Measures the amount and quality of output from Git commit history. Quantifies weekly features, added/modified lines, commit frequency, and other contributions. AI analyzes commit content to score how well requirements were implemented and how efficient the workflow was. Because this is collected and analyzed automatically in CI, nearly real-time feedback is possible.

Manager/Client Evaluation (Human Evaluation): The manager or client provides an overall evaluation. Each week (in under 25 minutes), they assess whether delivered features meet requirements, deliver business value, and whether there were process issues. They check things like “requirement alignment,” “user usability,” “priority of business-critical features,” and “communication quality.” Subjective satisfaction and field perspective—things AI cannot judge—are included here.

Theory and Aims Behind the 3-Axis Approach

The three-axis approach stems from the belief that a one-dimensional evaluation can’t capture an engineer’s true skill. If you measure only code quantity, you miss out on quality; if you look only at elegance, you lose track of productivity. Some value (such as user experience or teamwork) can’t be captured by numbers alone. By combining quality, quantity, and human evaluation, you get a balanced view.

The aims and theoretical foundations of this method are:

Eliminating Bias & Ensuring Fairness: Multiple axes prevent unbalanced evaluations. Because people have strengths and weaknesses, a combination of quality, quantity, and manager perspective ensures fairness. Combining automated (AI) and human review balances objectivity and human insight.
Short-Term Results and Long-Term Growth: By evaluating both deliverable quality (short-term) and quantity/process (long-term), the system encourages sustainable growth, not just one-off wins. This also encourages a culture of ongoing improvement.
AI-Era Appropriateness: In an age where AI writes code and generates tests, human value lies in “how well you use AI to create high-quality results efficiently.” CodeRanker quantifies what can be automated and leaves creativity and higher-order decisions to humans, reflecting this division in its design.
Transparency & Buy-In: Clear criteria and calculation methods for each axis are available to all. Everyone can see how their score is calculated and understand why they received a given evaluation, boosting trust and satisfaction.

Evaluation Flow and Score Integration

In CodeRanker, the results from the three axes are combined into a single integrated score for each engineer. The general flow is as follows:

Data Collection: Automatically collects project specs, test code, Git history, and CI test results.
Automated Evaluation (Deliverable Quality): Automatically checks requirements coverage, test results, and code security. Senior engineers also regularly review code with specialized tools (semi-automated, human-in-the-loop).
Automated Evaluation (Output Quantity): CI scripts analyze recent commits: number of features, lines changed, commit frequency/granularity, etc. AI also classifies commits (feature, bugfix, etc.) to score both quantity and content.
Manager/Client Evaluation (Human): The project manager reviews the auto-evaluation results and the actual product weekly, checking for requirement completeness, non-functional requirements, communication quality, and more.
Score Aggregation: Each axis's score is combined with predefined weights (default: Deliverable Quality 40%, Output Quantity 35%, Manager Evaluation 25%), producing an overall ranking.
Feedback: The final result, with explanations for each score and improvement recommendations, is provided to the engineer. They can use this feedback for personal growth and goal setting.

This flow repeats weekly or monthly, so the evaluation and feedback cycle stays close to real-time.

Benefits of the 3-Axis Evaluation System

Implementing the 3-axis system yields benefits not possible with traditional approaches:

Fair, Objective Evaluation: Using multiple axes prevents unfair, one-dimensional judgments. Combining data-driven automation and human review delivers fair, bias-resistant results.
Transparency: Scoring standards and calculation methods are shared, and everyone knows what is being measured. This increases buy-in and eliminates mistrust of “black box” HR decisions.
Real-Time Feedback: Output scoring happens in near real time, so engineers get instant feedback, can adjust quickly, and don’t have to wait for an annual review to improve.
Continuous Growth: By evaluating both output and process, engineers are encouraged to improve quality and efficiency over time, fostering a culture of self-improvement.
Human–AI Collaboration: The system automates everything possible, reducing burden on reviewers, while keeping human-only evaluations for context and nuance, building a truly human-centric system for the AI era.
Meritocracy: Scores and rankings make clear who delivers what, creating healthy tension and competition, rewarding results, and nurturing an achievement-oriented culture.

⸻

Summary: CodeRanker’s 3-axis system is a new framework that combines quality, quantity, and human review. It brings fairness and transparency to engineering evaluation for the AI era, supporting both individual and organizational growth. For CTOs and HR managers looking to modernize their evaluation systems, the 3-axis approach is a powerful solution.

Solving Management Issues with Payroll and Monthly Statistics Dashboards

CodeRanker’s system aggregates weekly evaluation points each month and publishes rankings and statistics for all members. Each engineer’s points are calculated based on the quality of deliverables, quantity of output, and manager (or client) satisfaction. Scoring follows a “Quality + Quantity + Satisfaction – Penalty + Bonus” formula, with documented standards and calculation methods. Weekly points are totaled monthly, and rankings are made by department and company-wide. This enables visualization of each member’s trends, team averages, and the distribution of high and low performers—helpful for managing and improving organizational performance.

Monthly rankings aren’t just for show—they directly impact pay. CodeRanker links evaluation results and compensation, so higher ranks get bonuses or raises, and lower ranks receive reductions or improvement programs. Each tier (T0–T7) has a base salary table, and monthly scores automatically determine adjustment. For example, exceeding a certain threshold earns a pay raise; falling below triggers a penalty. Calculation methods and rates are public, so everyone understands how pay is determined.

Key points for monthly stats and pay linkage:

Automatic Aggregation and Publication: Weekly points are summed each month and shared company-wide. This keeps performance transparent and visible.
Immediate Reward Reflection: Monthly rankings affect next month’s salary. Top performers get bonuses/raises, while underperformers get reductions or improvement plans—effort directly impacts pay.
Automated Calculation: Scoring is standardized, so rankings and salary adjustments are automatic, reducing admin burden.
Fairness and Buy-In: Criteria and methods are public, so everyone can check monthly stats and pay changes, ensuring trust and transparency.

Currently, fully automated CI/CD-driven monthly stats and payroll features are not yet live, but the architecture for weekly-to-monthly aggregation and payroll adjustment is in place. Full automation will bring even more efficiency in future updates.

Product

With the rapid advancement of AI, engineer evaluation systems also need to evolve. CodeRanker’s 3-axis system—grounded in fairness and transparency—offers valuable insight for many organizations.

For details, see the CodeRanker official website.

Source Code

[Starter Kit No.2] Exposing general-purpose Web scraping tool with text analysis function (automatic tagging / visualization)

makotunes — Mon, 02 Mar 2020 07:29:42 +0000

Easy Customizable Scraper

Concept

General-purpose Web scraping tool with text analysis function.

The following features help users start development.

Easy settings
Customizability
Text analysis function (tagging / visualization)

Click here for source code
https://github.com/makotunes/easy-customizable-scraper

Application example

Collection of curated media articles and automatic tagging
Recommendation engine

This algorithm is used for the functions of my personally developed products.
https://mockers.io/scanner

Elemental technology

Web scraping
Automatic language detection
Morphological analysis
Feature tagging algorithm (original)
2D map visualization technology (original)

Dependencies

Docker

Installation

It takes about 1-2 hours.

docker build -t scanner .

./build.sh

Run

docker run --rm -it -v "$PWD":/usr/src/app \
--name scanner --force scanner  \
-e 'ENTRY_URL=http://recipe.hacarus.com/' \
-e 'ALLOW_RULE=/recipe/' \
-e 'IMAGE_XPATH=//*[@id="root"]/div/div/section/div/div/div[1]/figure/img' \
-e 'DOCUMENT_XPATH=//td/text()|//p/text()' \
-e 'PAGE_LIMIT=2000' \
-e 'EXCLUDE_REG=\d(年|月|日|時|分|秒|ｇ|\u4eba|\u672c|cm|ml|g|\u5206\u679a\u5ea6)|hacarusinc|allrightsreserved' \
scanner:latest /usr/src/app/entrypoint.sh

./run.sh

Parametes

Set Environment Variable of Docker Container.

If you have at least ENTRY_URL, it will automatically scan the page and pull out the text.
If no options are specified, it is optimized for curated media and can be fully automated, such as extracting the text of articles.

Environment Variable	Description
ENTRY_URL	(Required) Site top URL to start scanning. All the pages are automatically scanned.
ALLOW_RULE	Allow filter rule of target urls.
DENY_RULE	Deny filter rule of target precedence overurls.
IMAGE_XPATH	Specify the image you want to get on the page with XPATH.
DOCUMENT_XPATH	XPATH of the top node in the page where text is to be extracted.
PAGE_LIMIT	Scaned limittation of number of pages. -1 means unlimited number.
EXCLUDE_REG	Regular expression of word rule not extracted by morphological analysis.

Result

result/res.json

Project structure

File	Description
src/scraper.py	Main scrapying logic
src/categorizer.py	Main algorithm to tag and visualize passages.
src/tokenizer.py	Main algorithm to do morphological analysis

Customize

custom/_formatter.py

Edit XPATH for required HTML nodes like below.

def formatter(sel):
    res = {}

    n_howtomake = int(len(sel.xpath('//*[@id="root"]/div/div/section/div/div/div[2]/div[1]/table[2]/tbody/tr/td/text()').extract()) / 2)
    res["n_howtomake"] = n_howtomake

    return res

custom/_finalizer.py

Edit post-process to generate your expected output like below.

import pandas as pd

def finalizer(res):
    pages = res["scatter"]
    pages = list(map(lambda x: x["user_meta"], pages))
    df = pd.DataFrame(pages)

    corr_df = df.loc[:, ["time", "n_howtomake", "n_components"]].corr()

    res["analyzed"] = {}
    res["analyzed"]["correlation"] = {}
    res["analyzed"]["correlation"]["time-n_howtomake"] = corr_df.loc["time", "n_howtomake"]

    return res

Example

Here is an example of using this tool for scraping and analysis.

Let's analyze free cooking recipe site.
** If you can't access it, try opening it in secret mode. **

All these results are stored in the "result" directory.

Visualize distribution of cooking time, number of ingredients, number of recipes

import matplotlib.pyplot as plt
import pandas as pd

def finalizer(res):

    pages = res["scatter"]
    pages = list(map(lambda x: x["user_meta"], pages))
    df = pd.DataFrame(pages)

    fig = plt.figure()
    x = df["time"]
    y = df["n_howtomake"]
    plt.scatter(x, y)
    plt.savefig('./result/time-n_howtomake.png')

    fig = plt.figure()
    x = df["time"]
    y = df["n_components"]
    plt.scatter(x, y)
    plt.savefig('./result/time-n_components.png')

    fig = plt.figure()
    x = df["n_howtomake"]
    y = df["n_components"]
    plt.scatter(x, y)
    plt.savefig('./result/n_howtomake-n_components.png')

    return res

Result

Cooking time: how to make

Cooking time: number of ingredients

How to make: Number of ingredients

Examine the correlation between cooking time, number of ingredients, and number of recipes

import matplotlib.pyplot as plt
import pandas as pd

def finalizer(res):
    pages = res["scatter"]
    pages = list(map(lambda x: x["user_meta"], pages))
    df = pd.DataFrame(pages)

    corr_df = df.loc[:, ["time", "n_howtomake", "n_components"]].corr()

    res["analyzed"] = {}
    res["analyzed"]["correlation"] = {}
    res["analyzed"]["correlation"]["time-n_howtomake"] = corr_df.loc["time", "n_howtomake"]
    res["analyzed"]["correlation"]["time-n_components"] = corr_df.loc["time", "n_components"]
    res["analyzed"]["correlation"]["n_howtomake-n_components"] = corr_df.loc["n_howtomake", "n_components"]

    fig = plt.figure()

    return res

Result

Cooking time: how to make

0.30457219729662316

Cooking time: number of ingredients

0.3949520467754227

How to make: Number of ingredients

0.6869899620517819

Select about three keywords that can be used to characterize each Leshig

result/tagged.csv

title	tag1	tag2	tag3
なすとトマトの中華和え(１５分)	なす	トマト	大葉
ぶりの照り焼き(45分)	両面	照り焼き	水気
おでん風煮(2時間)	大根	こんにゃく	竹輪
大根とツナのサラダ(15分)	ツナ	大根	わかめ
鶏の照り焼き丼(20分)	片栗粉	にんにく	れんこん
筑前煮(６０分)	れんこん	ごぼう	こんにゃく
白菜とわかめの酢の物(15分)	白菜	わかめ	しめじ
鮭のホイル焼き(25分)	玉ねぎ	しめじ	ピーマン
キャベツとハムの粒マスタード和え(１５分)	キャベツ	しめじ	ハム
えのきとワカメの和え物(15分)	えのき	わかめ	きゅうり
里芋のおやき(３０分)	里芋	片栗粉	桜えび
鶏肉と里芋の煮物(60分)	里芋	鶏肉	相性
焼き万願寺唐辛子(10分)	万願寺唐辛子	かつお節	作り方
肉じゃが(45分)	玉ねぎ	牛肉	じゃがいも
鰆の幽庵焼き(１５分(漬け込む時間は省く))	冷蔵庫	鰆	ゆず
オクラの煮びたし(１０分(冷やす時間は除く))	オクラ	しょうが	オクラ
春菊と油揚げの煮びたし(15分)	春菊	油揚げ	春菊
しいたけのツナマヨ焼き(１０分)	マヨネーズ	塩コショウ	しいたけ
コールスロー(15分)	キャベツ	マヨネーズ	きゅうり
じゃがいもとタコのガーリック炒め(２０分)	にんにく	じゃがいも	タコ
棒棒鶏(３０分)	しょうが	トマト	きゅうり
アボカドのチーズ焼き(15分)	アボカド	ハム	チーズ
キャベツと大葉のさっぱり和え(１０分)	大葉	キャベツ	大葉
カニカマサラダ(10分)	レタス	きゅうり	サラダ
ズッキーニともやしのナムル(１５分)	ズッキーニ	もやし	粗熱
春雨サラダ(１５分)	春雨	きゅうり	ハム
白菜と油揚げのみぞれ煮(３０分)	大根	白菜	油揚げ
牛肉とれんこんの甘辛炒め(30分)	れんこん	にんにく	牛肉
豚丼(30分)	しょうが	しめじ	レタス
紅白なます(30分)	ゆず皮	大根	部分
里芋のガーリック焼き(３０分)	里芋	にんにく	香り
ブロッコリーのごまみそ和え(10分)	ブロッコリー	和風	みそ
ブロッコリーのゴマ和え(１５分)	ブロッコリー	出汁醤油	粗熱
かぶの甘酢漬け(１時間)	ゆず	甘酢	昆布
さんまのしょうが煮(３０分)	さんま	しょうが	圧力
スパゲティーサラダ(２０分)	大根	きゅうり	スパゲッティ
切り干し大根の煮物(２５分)	切り干し大根	油揚げ	短冊
なすとオクラの和え物(１０分)	なす	オクラ	出汁醤油
なすと厚揚げのおろしあん(30分)	厚揚げ	なす	片栗粉
ごぼうのごまマヨサラダ(15分)	ごぼう	好み	一味
水菜と長いものわさび和え(１５分)	水菜	長いも	わさび
ピーマンのじゃこ炒め(１５分)	ピーマン	雑魚	顆粒和風だし
なすと豚肉のごまみそ丼(20分)	なす	丼	ピーマン
ピリ辛豆腐ステーキ(30分)	豆腐	しょうが	白ネギ
白菜とハムの青じそサラダ(20分)	白菜	大葉	ハム
鮭のシャリアピンソースがけ(30分)	鮭	にんにく	ソース
白菜のさっぱりサラダ(15分)	白菜	きゅうり	サラダ
かぶと肉団子の煮物(30分)	しょうが	鶏ミンチ	片栗粉
ごぼうの梅おかか煮(４５分)	ごぼう	かつお節	圧力鍋
切り干し大根とほうれん草の和え物(20分(水に戻す時間は除く))	ほうれん草	切り干し大根	熱湯
かぼちゃと揚げの煮物(20分)	かぼちゃ	油揚げ	揚げ
さつまいものレモン煮(30分)	さつまいも	レモン汁	レモン
菜の花の辛子和え(15分)	菜の花	練りからし	長さ
かぼちゃとこんにゃくの煮物(30分)	こんにゃく	かぼちゃ	熱湯
ゆず入り湯豆腐(１時間)	豆腐	春菊	好み
大根と厚揚げの煮物(６０分)	厚揚げ	大根	彩り
小松菜ぎょうざ(45分)	小松菜	にんにく	しょうが
水菜と油揚げの煮びたし(15分)	油揚げ	水菜	食感
ふろふき大根(３０分以上)	大根	味噌	いりごま
春菊の白和え(１５分)	春菊	豆腐	白和え
なすのホイル焼き(15分)	なす	生姜	ホイル
えびとニラの中華風卵炒め(30分)	玉ねぎ	ニラ	えび
ししゃもの南蛮漬け(３０分)	ししゃも	南蛮漬け	ピーマン
八宝菜(３０分)	豚肉	白菜	玉ねぎ
ブロッコリーのわさマヨ和え(１５分)	ブロッコリー	食感	わさび
鶏のすき煮(30分)	鶏肉	鶏もも肉	しいたけ
里芋の梅おかか和え(35分)	里芋	梅干し	かつお節
ブロッコリーの磯和え(15分)	ブロッコリー	出汁醤油	焼き海苔
焼き鳥丼(20分)	鶏肉	鶏もも肉	白ネギ
ほうれん草のお浸し(10分)	かつお節	ほうれん草	10分
さんまの梅しそロール(45分)	片栗粉	さんま	大葉
きゅうりとトマトの土佐酢和え(３０分)	トマト	きゅうり	かつお節
さばの味噌煮(30分)	さば	味噌	しょうが
エリンギのバター炒め(15分)	エリンギ	バター	エリンギ
さつまいもとクリームチーズのサラダ(２0分)	さつまいも	マヨネーズ	塩コショウ
薄揚げの納豆キムチ詰め(15分)	キムチ	納豆	長ネギ
鮭の味噌ヨーグルト漬け(１５分(漬け込む時間は除く))	味噌	鮭	ヨーグルト
五目豆(４５分)	ごぼう	れんこん	こんにゃく
新生姜と水菜の肉巻き(３０分)	新生姜	水菜	肉
小松菜とツナの和え物(１５分)	小松菜	ツナ	水気
手羽中と大根の煮物(４５分)	大根	手羽中	弱火
手羽先の照り焼き(60分)	表面	手羽先	にんにく
ほうれん草の梅和え(15分)	ほうれん草	梅	梅干し
なめこおろし(１５分)	なめこ	大葉	かつお節
簡単タンドリーチキン(１時間以上)	1時間	塩コショウ	タンドリーチキン
あさりの酒蒸し(15分)	あさり	にんにく	みじん切り
ピーマンの肉詰め(３０分)	玉ねぎ	ピーマン	肉
ちくわの磯辺揚げ(15分)	竹輪	青のり	衣
長いもの梅和え(10分)	長いも	かつお節	梅干し
水菜とアボカドのサラダ(１５分)	アボカド	水菜	豆腐
酢鶏(20分)	鶏肉	鶏がらスープの素	一口
鯛の西京焼き(１５分(漬け込む時間は除く))	鯛	冷蔵庫	魚
きゅうりの塩昆布和え(10分)	きゅうり	塩昆布	乱切り
なすの煮びたし(20分)	なす	しょうが	作り方
ごぼうと人参の肉巻き(30分)	ごぼう	肉	にんにく
大根・里芋・イカの煮物(40分)	イカ	大根	里芋
回鍋肉(30分)	豚肉	にんにく	片栗粉
ほうれん草と干しえびのゴマ和え(１５分)	ほうれん草	干しエビ	干しえび
里芋のそぼろ煮(３０分)	里芋	片栗粉	鶏ミンチ
三度豆と人参のおかか和え(１０分)	三度豆	湯	出汁醤油
中華丼(30分)	チンゲン菜	しめじ	豚肉
きゅうりとたこの酢の物(20分)	きゅうり	たこ	わかめ
新玉ねぎのコンソメ煮込み(45分)	片栗粉	鶏ミンチ	新玉ねぎ
れんこんのきんぴら(１５分)	れんこん	いりごま	中火
野菜たっぷり牛丼(20分)	玉ねぎ	ニラ	しめじ
ホタテとチンゲン菜のクリーム煮(２０分)	片栗粉	にんにく	チンゲン菜
ほうれん草とごぼうの白和え(60分)	ごぼう	豆腐	ほうれん草
さんまの蒲焼き(３０分)	さんま	ごま	大葉
ひじきの炒め煮(６０分)	ひじき	油揚げ	大豆
オクラの納豆和え(１０分)	かつお節	オクラ	納豆
じゃこのサラダ(１０分)	縮緬雑魚	貝割れ大根	水菜
里芋のホットサラダ(45分)	里芋	ほうれん草	ベーコン
かぼちゃのサラダ(１５分)	かぼちゃ	ヨーグルト	きゅうり
豆腐のきのこあんかけ(１０分)	豆腐	しめじ	えのき
春巻き(60分)	しょうが	春雨	ニラ
れんこんのはさみ焼き(３０分)	れんこん	しょうが	鶏ミンチ
高野豆腐の含め煮(30分)	高野豆腐	竹串	水気
豚ミンチと白菜の炒め物(15分)	春雨	白菜	にんにく
いわしのさっぱり煮(４５分)	いわし	しょうが	長ネギ
れんこんのカレー炒め(１５分)	れんこん	OLIVE OIL	カレー粉
ささみの中華風サラダ(25分)	ささみ	もやし	きゅうり
茄子と豚肉のピリ辛味噌炒め(30分)	豚肉	茄子	豆板醤
ささみのから揚げ(30分)	ささみ	にんにく	しょうが

Call for ideas

What do you want to do with this tool?
If there is a need, it may be addressed in a future update.
We look forward to your comments.

[GPT-2] The problem feeling when trying to support multi-language with GPT-2

makotunes — Sat, 18 Jan 2020 14:11:34 +0000

Online tool is published

Normal GPT-2 only supports English, but developed a tool that supports multiple languages.

https://mockers.io/generator

Design

Actually it has a three-stage configuration,

Language identification and translation into English
GPT-2 main processing
Restore from English to the identified original language

It is a fake multilingual support.

The model is compatible with the recently published best performing 1558M, which seems to produce very natural sentences in English.
On the other hand, in Japanese, it is a very machine-translated sentence.

Problem

It will be more natural to support Japanese native as a model.

GPT-2 originally uses English-written news sites as learning data, so it is good at generating sentences that can be told there. GPT-2 basically gives a sentence first and then guesses the proper sentences to come after. For example, if you generate "President Trump" in Japanese, it will be a bit decent. Other Languages probably too.

On the other hand, giving words that are far from English-speaking cultures tends to be unnatural. In order to obtain natural Japanese, of course, we do not use translations, but it is also very important to learn Japanese-speaking sentences.

By the way, it is possible to generate sentences recursively from the generated sentences, but it is also a problem that in languages other than English, the translation is repeated and it deteriorates rapidly.

But is there a need for multilingual native models?

Prospects for future AI generation

At present, sentence generation in large-scale unsupervised learning has become higher in terms of naturalness, but I feel that it is not good to adapt to something.
In my personal opinion, I have the following issues.

How to learn common sense and facts
Can I generate sentences with purpose?

If these parts are resolved, I think it will have business value.

Finally

GPT-2 related tools are highly acclaimed and under development, so if you are interested, please read this article.

https://dev.to/makotunes/by-using-gpt-2-which-is-too-dangerous-i-finetuned-a-model-with-president-trump-s-twitter-and-made-a-fake-trump-bot-2j0h

[GPT-2] Fine-tuning a model with President Trump's Twitter and making a fake Trump bot with GPT-2 , which is "too dangerous"

makotunes — Sat, 21 Dec 2019 10:31:51 +0000

Background

By using GPT-2, which is published by OpenAI, We can generate very natural passages.

According to Cornell University survey, some say that 70% of people who read the text generated by GPT-2 misunderstand the text as an article in the New York Times.

At first, full size model was not published because developers guessed that it would have risk used with bad purpose.
However, fortunately, the full size model (the largest 1.5 billion class) model has been already released and availeble.

However, in fact, we are not sure about how this AI can be used effectively.
Therefore, I developed and released a web application called "Mockers", an online tool that anyone can easily use GPT-2.
This will provide an opportunity to consider how to use GPT-2.

First of all, if you want to see what GPT-2 is, try this Mockers generation tool.
https://mockers.io/generator

Please refer to below links to understand how to use.
https://doc.mockers.io/archives/1966/
https://doc.mockers.io/archives/1987/

Purpose

Share the results of experiments using Mockers for fine-tuning.

Fine-tuning is to provide additional data using a model that has already been learned, perform learning at low cost, and generate another model.
A model is created that learns the context and style of the given sentence and generates the sentence in a way that follows.
Mockers does not just try GPT-2, but also supports fine-tuning and automatic posting.

Usecase

By using this mechanism, for example, the following use cases are realized.

-You can build a media that receives Page Views outbreaks so that it does not infringe on the copyright of a curated media, imitate it, and parasitize it.

-You can build a bot that keeps impersonating a certain Twitter account.

Experiment

In this article, as a demonstration, to experiment with fine-tuning using GPT-2,
I used Mockers to finetunin about President Trump's Twitter and create a President Fake Trump bot.

Here, too, you can always see the latest President Trump's Mock.
https://mockers.io/timeline

Procedure

Please visit here.
https://mockers.io/login

You need to login for fine-tuning. Sign up or use a Google account.

If the login is successful, you will be prompted to create a model, so press “Create screen”.

When the new model dialog is displayed, input "Model name" as appropriate and set "Model type" to "Custom model (Twitter)". This allows you to generate a fine-tuning model for your Twitter account. On "Model Settings", In "Sync Target Account (input)", enter the target Twitter account.

Currently, it takes up to two hours to generate a model.
When a model is generated, it is automatically generated periodically, but you can also register an account to tweet the generated text.
To do this, you need to register with Twitter API in advance.
See the following article for how to apply.
https://dev.to/twitterdev/using-the-twitter-api-to-make-your-commute-easier-3od0

The Twitter account entered in "Sync Target Account" triggers original your account to generate a sentence that is related to the tweet to Twitter.

In this way, "mock" is realized.

Result

Here's what Fake Trump actually tweeted: It's not necessarily what he says, but sometimes he says something appropriate to his position, and otherwise it's generated with topics related to what he said in the past You can see that.

Problem

GPT-2 can control the length of each word, but cannot control the number of characters, so it cannot be optimized with media that is severe in character length such as Twitter. As a result, if you exceed the Twitter limit of 280 characters, you have to force cut it.
The GPU memory required for fine-tuning is too large, and models larger than 774M will not work with GPUs that can be used by ordinary people. Even the personally owned "Geforce GTX1080 Ti" and AWS P3 instance "Tesla V100" did not work due to lack of memory. (Normal prediction can work well)

In the end

It is expected that sentence generation technology using large-scale unsupervised learning will continue to improve its accuracy and shift to the phase of practical use in the future. We hope this article and Mockers can contribute to natural language AI, development and social implementation at all.

P.S.
Don't forget Hillary.

https://mockers.io