DEV Community: Eana Hufwe

Rotated Background Patterns in CSS with SVG

Eana Hufwe — Thu, 06 Mar 2025 21:18:49 +0000

While working on the WordPress theme Tìtsyul Amip Twenty Twenty-Five, a need arose for a more flexible way to implement rotated background patterns in CSS. Existing methods often involve trade-offs, and this exploration led to a technique leveraging the power of SVG.

Existing Approaches and Their Limitations

Traditionally, achieving rotated background images in CSS has involved several workarounds, each with its own set of challenges:

CSS Transforms: Using the transform: rotate() property. This rotates the entire element, including its content, which is often not the desired outcome.
Pseudo-elements: Employing ::before or ::after pseudo-elements to create a rotated background. While this allows rotating the background independently of the content, it can be complex to manage sizing and positioning. Additionally, if the html or body elements have a background color, pseudo-elements with a negative z-index may fall behind them.
Pre-rotated Images: Rotating the image using image editing software and then using the rotated image as the background. This is impractical for seamless patterns in most rotation angles or when dynamic canvas size is required.

The SVG Pattern Technique

Building on these insights, let’s now explore a method that leverages SVG to achieve rotated background patterns—a solution that delivers both flexibility and precise control.

At the core of this approach is the SVG <pattern> feature, a versatile tool that enables the creation of seamless, repeating background patterns. The <pattern> element defines a tileable area that can be used as a fill for other SVG shapes, allowing for intricate and scalable designs. By leveraging attributes such as patternTransform, you can easily manipulate these patterns with transformations like rotation, scaling, and translation, making it an ideal choice for dynamic and responsive backgrounds.

Draw the Pattern Using SVG ⸻ Begin by creating the desired pattern using SVG elements. This can include vector graphics or embedded bitmaps.

<polygon points="0,0 2,5 0,10 5,8 10,10 8,5 10,0 5,2" />

Wrap in a <pattern> Definition ⸻ Encapsulate the pattern within a <pattern> element inside the <defs> section of the SVG, specify the dimensions for the pattern, and use the patternTransform attribute to apply transformations, such as rotation.

<defs>
  <pattern id="star" width="10" height="10" patternUnits="userSpaceOnUse" patternTransform="rotate(20)">
    <polygon points="0,0 2,5 0,10 5,8 10,10 8,5 10,0 5,2" />
  </pattern>
</defs>

Create the Main SVG ⸻ Create a new SVG element without a viewBox attribute. Set the width and height to 100% to ensure it covers the entire area of the element. Then add a <rect> element to cover the entire SVG area, and set its fill attribute to reference the pattern defined in step 2.

<svg xmlns="http://www.w3.org/2000/svg" width="100%" height="100%">
  <defs>
    <pattern id="star" width="10" height="10" patternUnits="userSpaceOnUse" patternTransform="rotate(20)">
      <polygon points="0,0 2,5 0,10 5,8 10,10 8,5 10,0 5,2" />
    </pattern>
  </defs>
  <rect width="100%" height="100%" fill="url(#star)" />
</svg>

Encode and Use in CSS ⸻ Encode the SVG as a data URL. Use the data URL as the background-image in your CSS.

div {
  background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='100%25' height='100%25'%3E%3Cdefs%3E%3Cpattern id='star' width='10' height='10' patternUnits='userSpaceOnUse' patternTransform='rotate(20)'%3E%3Cpolygon points='0,0 2,5 0,10 5,8 10,10 8,5 10,0 5,2' /%3E%3C/pattern%3E%3C/defs%3E%3Crect width='100%25' height='100%25' fill='url(%23star)' /%3E%3C/svg%3E");
}

Using this technique, the pattern background can be transformed on its own, while still behave like any other CSS background to enjoy other features like background stacking and blend mode settings.

This is an example of how a background using this technique could look like.

SVG Rotated Background Generator

You can also try it out using the generator below to create custom rotated background patterns with SVG. You can also adjust parameters such as pattern size, scaling, and rotation angle.

The post Rotated Background Patterns in CSS with SVG appeared first on 1A23 Blog.

Shift-JIS / UTF-8 文字化け解読：実はもうちょっと読めるかも

Eana Hufwe — Mon, 01 Apr 2024 04:49:43 +0000

文字化け、それは一昔前のパソコンにおいて情報交換でよく起こる不具合である。近年になってからは Unicode（だいたい UTF-8）がネット上の文字コードの事実上の標準になっており、意図しないで発生する文字化けはほとんどみられなくなった。今では、よくみられる文字化けはほぼ創作でホラー要素、謎解き、あるいは隠しメッセージとして登場している。その中に、最もよく使われている文字化けの種類は「Shift-JiS / UTF-8」による文字化けで、いわゆる「繝繧」とか糸へんの漢字がいっぱい入ってるのやつ。この記事は、「Shift-JiS / UTF-8」による文字化けを既存ツールよりはもうちょっと解読できる方法を解説します。

Dev.to に一部内容が表示できないため、原文を参照してください。
原文

どれくらい解読できる？

まず実際の解読例を見てみましょう。

元の文章	文字化けした日本語のテキストの例
化けた文字列	譁�蟄怜喧縺代＠縺滓律譛ｬ隱槭�ｮ繝�繧ｭ繧ｹ繝医�ｮ萓�
既存のツールで解読	�字化けした日本語��キスト��
今回のツールで解読 (JIS 第一水準漢字限定)	(文斉斌斎斐斑斗料斜)字化けした日本語(の)(ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミム)キスト(の)(侃例侍供依侠)

この例の通り、既存のツールで解読した「?」の部分に入りそうな文字の候補を提示してくれるのが今回のツールです。では、いったいどうやって候補を検出できるのかを解説していこう。

文字コードの前提知識

細かい仕組みに踏み入る前に、まずは今回扱う2種類の文字コード、Shift-JIS と UTF-8 の構造を少し見てみよう。

Shift-JIS は相対的に単純で、一文字は１バイトもしくは２バイトで構成して、それぞれで使える値の範囲が限られている。次の表の通り、１ビットは必ず 0x00–0x1F 0x20–0x27 0xA1–0xDF のどれかで、２バイトの場合は、１バイト目は 0x81–0x9F 0xE0–0xFC で、２バイト目は 0x40–0x9E 0x9F–0xFC に入ることになる。

Dev.to に色づきの表は挿入できないため、Wikipedia にて Shift-JIS 拡張の符号表を参照してください。

対して、UTF-8 が対応する範囲がかなり広いので、符号化の体系もより複雑になっている。U+0000 から U+10FFFF までの約 111 万の符号位置を表す必要があるため、UTF-8 が使用するバイト幅が１バイトから４バイトまでになる。

セル結合した表は Dev.to に表示できないため、原文を参照してください。
原文

パターン的に、１バイトから４バイトの構成ではそれそれ先頭のビットは確定されている。

実際に解読する

Shift-JIS でエンコードされた文字列を UTF-8 でデコードすると、対応する文字が見つからない場合があり、それが文字化けの原因となる。先の例文を使って実際エンコードするとこうなります。

セル結合した表は Dev.to に表示できないため、原文を参照してください。
原文

上の表の�の部分は、データを Shift-JIS として解読する時にデータに対応する文字が見つからないため、仮に置かれるプレースホルダーである。元データと照らし合わせると、「テ」「の」「例」に対応するデータの一部が復元できないことがわかる。このため、一般の文字化け解読ツールは「�キスト��」までしか解読できない。

ただし、この場合で失ったデータは１文字ごとに１バイトだけ、残った部分はまだ解読のヒントになれるはず。いくつかの仮定を立てて、この表をより細かく分けてみよう。

仮定その１：元データは有効な UTF-8 データ

セル結合した表は Dev.to に表示できないため、原文を参照してください。
原文

タイトルに書いた通り、当然元データは UTF-8 でなかったり、改変したりすると、そもそも大前提が成り立たない。この仮定のもとに、復元できない部分を UTF-8 の先頭ビット規則に当てはまると、実際に失われたデータはこの場合 8 ビットずつではなく 6 ビットずつになる。

1 ビット、つまり二進数の一桁は 0 か 1 の二通りで構成するから、6 ビットの組み合わせは 2⁶ = 64 通りになる。これで無限にある可能性から一文字分 64 個の候補に絞られます。

この仮定で絞られた候補はこちら：

一文字目・二進数の 0011000011 ______ に当てはまる Unicode 文字：

ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶヷヸヹヺ・ーヽヾヿ

二文字目・二進数の 0011 ______ 101110 に当てはまる Unicode 文字：

〮のギヮㄮㅮㆮ㇮㈮㉮㊮㋮㌮㍮㎮㏮㐮㑮㒮㓮㔮㕮㖮㗮㘮㙮㚮㛮㜮㝮㞮㟮㠮㡮㢮㣮㤮㥮㦮㧮㨮㩮㪮㫮㬮㭮㮮㯮㰮㱮㲮㳮㴮㵮㶮㷮㸮㹮㺮㻮㼮㽮㾮㿮

三文字目・二進数の 0100111110 ______ に当てはまる Unicode 文字：

侀侁侂侃侄侅來侇侈侉侊例侌侍侎侏侐侑侒侓侔侕侖侗侘侙侚供侜依侞侟侠価侢侣侤侥侦侧侨侩侪侫侬侭侮侯侰侱侲侳侴侵侶侷侸侹侺侻侼侽侾便

つまり、今のところの解読予想はこうである

ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶヷヸヹヺ・ーヽヾヿ	キ	ス	ト	〮のギヮㄮㅮㆮ㇮㈮㉮㊮㋮㌮㍮㎮㏮㐮㑮㒮㓮㔮㕮㖮㗮㘮㙮㚮㛮㜮㝮㞮㟮㠮㡮㢮㣮㤮㥮㦮㧮㨮㩮㪮㫮㬮㭮㮮㯮㰮㱮㲮㳮㴮㵮㶮㷮㸮㹮㺮㻮㼮㽮㾮㿮	侀侁侂侃侄侅來侇侈侉侊例侌侍侎侏侐侑侒侓侔侕侖侗侘侙侚供侜依侞侟侠価侢侣侤侥侦侧侨侩侪侫侬侭侮侯侰侱侲侳侴侵侶侷侸侹侺侻侼侽侾便

仮定その２：元の文字列は日本語である

さっきの仮定で 64 個の候補に絞られたが、並べるとまだ多すぎる気がする。ここで2つ目の仮定の登場です。原理的に UTF-8 でエンコードされたデータが Shift-JIS でデコードするだけだから、どんな言語でもできるが、「そもそも日本語じゃないと Shift-JIS で化ける必要なくない」と思っちゃうから、ここで原文が日本語であることを仮定する。

上のリストに「ㅮ」や「侦」など日本語じゃほぼ絶対使わない文字が散見するが、文字単位で十分な言語判定はできないから、ここで大まかに Shift-JIS に符号化できる文字のみを「日本語」とします。

この条件で絞られた候補は以下のとおりである。かなり少なくなったね。

ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶ・ーヽヾ	キ	ス	ト	のギヮ	侃來侈例侍侏侑侖侘供依侠価侫侭侮侯侵侶便

仮定その３：化け戻すとデータはロストしなければならない

タイトルはちょっと回りくどいが、まず以下の例を見てみよう。一文字目の候補の「ム」と「メ」の2文字に注目して、それぞれが当てはまる場合再び文字化けをする時の結果を求めよう。

「ム」の場合

予想の原文：ムキスト…
UTF-8 でエンコード：E3 83 A0 E3 82 AD …
Shift-JIS でデコード：繝�繧ｭ…
解読したい文字列：繝�繧ｭ…

「メ」の場合

予想の原文：メキスト…
UTF-8 でエンコード：E3 83 A1 E3 82 AD …
Shift-JIS でデコード：繝｡繧ｭ…
解読したい文字列：繝�繧ｭ…

一文字目に「ム」を代入して予想の原文を化け直すと最初から解読したい文字列に一致する結果が得られるが、「メ」の場合は化け直す文字列に�が含まれていない。つまり、背理法（?）によって「メ」は一文字目ではないことが証明できる。この方法で全ての候補を化け直してチェックすると、候補をさらに絞られます。

ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミム	キ	ス	ト	の	侃來侈侊例侍侏侑侒侔侖侘侚供依侠

これで５文字目が「の」であることを確定できました。

仮定その４：難しい漢字は使わない（任意）

さっきの候補はすでにかなり絞られたが、中にはまだ「侈」や「侊」など普通じゃ読めない漢字がかなり残っている。解読したい文字列の出自にもよるが、難読漢字は出ない場合は多いだろうと仮定する。ここは上の候補の中にある漢字を JIS 第一水準漢字に絞ると、こうしてさらに候補を減らせる。

ダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミム	キ	ス	ト	の	侃例侍供依侠

当然、自信があれば常用漢字や教育漢字などの文字集合に絞るのもありです。

補足：１バイトと２バイト⸺デコーダ仕様の違い

最後に補足したいのは、文字化けはツール（デコーダ）の仕様によって結果はかならず同じではないこと。私はいろんな文字化けツールを試している中、一つ面白いことが気付いた。それは、ツールによっては同じ文章でも、文字化けにすると違う結果が出て、それぞれデコードできないデータの処理方法が違うことである。ここでデコーダをその仕様で「１バイト類」と「２バイト類」に分ける。

「１バイト類」のデコーダは解読できないデータを見つけたら、１バイト飛ばして次から解読するものである。例えば、「E3 83 86 E3 82 AD」をデコードするときはこういう風になる：

E3 ← 2バイト文字の第1バイト、次のバイトを見る
E3 83 ← 繝
繝 86 ← 2バイト文字の第1バイト、次のバイトを見る
繝 86 E3 ← 対応する文字がない、次のバイトから再開
繝� E3 ← 2バイト文字の第1バイト、次のバイトを見る
繝� E3 82 ← 繧
繝�繧 AD ← ｭ
繝�繧ｭ ← 結果

対して、「２バイト類」のデコーダは解読できないデータを見つけたら、解読できない部分をまとめて飛ばすものである。例えば、同じく「E3 83 86 E3 82 AD」をデコードするときはこういう風になる：

E3 ← 2バイト文字の第1バイト、次のバイトを見る
E3 83 ← 繝
繝 86 ← 2バイト文字の第1バイト、次のバイトを見る
繝 86 E3 ← 対応する文字がない、まとめて飛ばす
繝� 82 ← 2バイト文字の第1バイト、次のバイトを見る
繝� 82 AD ← く
繝�く ← 結果

このため、デコーダによっては、�が１バイトを意味する時があるし、２バイトを意味する時もある。冒頭にあるツールで「?」は１バイトを意味するため、�を１バイトや２バイトに変換するボタンを用意した。片方が解読できなかったら、もう一方も試してみよう。

これからさらに絞りたいなら、たぶん文脈で頑張るしかないかなと思う。この方法で、最初の無限にある選択肢からかなり限られたオプションに絞られて、文脈で当てる難易度も結構下げたと思う。

もしこの先にまたどこかで文字化けを解読したい時があったら、この記事・ツールが役に立つことができたらとても嬉しいです。ではまた。

The post Shift-JIS / UTF-8 文字化け解読：実はもうちょっと読めるかも appeared first on 1A23 Blog.

Reverse engineering an IL2CPP NSO binary: Case study of Mojipittan Encore

Eana Hufwe — Mon, 16 Jan 2023 23:29:35 +0000

This is yet another random side project I was working on recently, and my first attempt to reverse engineer a real world application compiled into binary. In this article, I want to talk about how I reversed engineered an Unity IL2CPP binary compiled to NSO, in a step-by-step fashion.

Forewords

Kotoba no Puzzle: Mojipittan is a word puzzle game series in Japanese, where the player makes words from letter pieces on a board. It sort of like Scrabble, but not exactly the same. I always wanted to give it a try, but it was quite an old game, available only on GBA, PSP, DS, and Wii. That’s until they made an Encore version on Nintendo Switch. This game is also one of the reasons I bought a Switch.

Ever since I bought the game, I was wondering if there is a way to solve the game optimally. The first step to approach this is to first get the dictionary of the game. Despite the game has now released to multiple different platforms, there has no resource online on the word list used in the game. I thus decided to do it myself.

Preparation

To follow this article, you would need an exploitable Nintendo Switch with a purchased copy of Mojipittan Encore and a PC running Windows. Although there are other ways to do it with only a PC, I do not recommended shem for reasons. You might be able to also use other OSes, but a lot of the resources has precompiled binary for Windows, which involves less effort.

To get the game data from the device, you need to dump it from the console. Yuzu has provided a detailed guide on how to dump games and corresponding keys. I will use an XCI dump here as an example.

Extract contents

To extract the dumped XCI file, I used hactool, specifically this wrapped version Unpackv2.

SciresM / hactool

hactool is a tool to view information about, decrypt, and extract common file formats for the Nintendo Switch, especially Nintendo Content Archives.

hactool

hactool is a tool to view information about, decrypt, and extract common file formats for the Nintendo Switch, especially Nintendo Content Archives.

It is heavily inspired by ctrtool.

Usage

Usage: hactool [options...] <file&gt
Options:
-i, --info        Show file info.
                      This is the default action.
-x, --extract     Extract data from file.
                      This is also the default action.
  -r, --raw          Keep raw data, don't unpack.
  -y, --verify       Verify hashes and signatures.
  -d, --dev          Decrypt with development keys instead of retail.
  -k, --keyset       Load keys from an external file.
  -t, --intype=type  Specify input file type [nca, xci, pfs0, romfs, hfs0, npdm, pk11, pk21, ini1, kip1, nax0, save, keygen]
  --titlekey=key     Set title key for Rights ID crypto titles.
  --contentkey=key   Set raw key for NCA body decryption.
  --disablekeywarns  Disables warning output when loading external keys.
NCA options:
  --plaintext=file   Specify file path for saving a decrypted copy of the NCA.
  --header=file      Specify Header file

…

View on GitHub

Once the tool is downloaded and unzipped, follow the next steps:

Copy the prod.keys extracted form your device to the folder where Unpack.cmd is found, and rename it to keys.txt.
Drop the dumped XCI file onto Unpack.cmd to start the script.
When prompted… If your patch was inside XCI, press "1" and ENTER If you don't have a patch, just only press ENTER …press Enter. This game exported did not come with a patch.
Now, in the Unpackv2 folder, there will be a new ExtractedXCI folder created with 4 NCA files of various sizes created. When prompted… Drop here correct NCA patch file (probably the biggest one) from ExtractedXCI folder in …, drop the 774MB a0547397496b93fcb08f438bcaad2731.nca to the terminal window.
Now the terminal print a list of files extracted, ending with the prompt… Press ENTER to delete all temporary files Press Enter twice to finish the export.

After finishing the export process, there will be a new folder created with the extracted content, with two subfolders: exefs and romfs. We will make use of these contents in the next step.

Unpack resource files

When you open the romfs/Data folder, you will be welcomed with some familiar file names, like resource.assets, sharedassets0.assets, level0, and level1. Yes, if you have ever made or opened the directory of a Unity game, you will surely recognize these file names. Unity organizes their asset files in a pretty recognizable pattern, and is well studied by the community with multiple tools created.

At this point, you are free to extract all static assets files found in the game, like text files and textures. The tool I used is Unity Asset Bundle Extractor (UABE). To extract assets, open the .assets files with the tool, and export the assets using the built-in plugins to easily processable formats.

SeriousCache / UABE

Asset Bundle Extractor

.assets and AssetBundle editor.
Not affiliated with Unity Technologies.

UABE is an editor for 3.4+/4/5/2017-2021.3 .assets and AssetBundle files. It can create standalone mod installers from changes to .assets and/or bundles.

There are multiple plugins to convert assets from/to common file formats :

The Texture plugin can export and import .png and .tga files (Texture2D only) and decode&encode most texture formats used by Unity.
The TextAsset plugin can export and import .txt files.
The AudioClip plugin can export uncompressed .wav files from Unity 5+ AudioClip assets using FMOD, .m4a files from WebGL builds and Unity 4 sound files.
The Mesh plugin can export .obj and .dae (Collada) files, also supporting rigged SkinnedMeshRenderers.
The Utility plugin can export and import byte arrays and resources (StreamingInfo, StreamedResource) within the View Data editor.

Building

UABE can be built within Visual Studio (Community) 2022 using the Open Folder option (CMake).

The…

View on GitHub

Besides the folder mentioned above, there is a subfolder StreamingData/Switch/datas, which contains assets that are loaded after the game has initialized. Here, the files we are interested are the dictionary files under romfs/Data/StreamingData/Switch/datas/dictionary. Open it with UABE, we can see three text assets: worddata.aid, worddata.cot, and worddata.dic. Extract them with the txt export plugin, we can get three binary files with some sort of patterns.

$ xxd worddata.aid | head -n 20
00000000: 5744 5000 0002 1ca1 0000 0001 0001 c1e6 WDP.............
00000010: 0000 0002 0001 ffee 0001 5eb2 0001 5eb3 ..........^...^.
00000020: 0000 0003 0001 ffef 0001 fff0 0000 0004 ................
00000030: 0000 0005 0000 0006 0001 5d48 0000 0007 ..........]H....
00000040: 0000 0008 0000 0009 0000 000a 0000 000b ................
00000050: 0001 92b4 0001 92b7 0001 92b3 0001 92b6 ................
00000060: 0001 92b5 0000 000c 0001 5a5f 0001 fff1 ..........Z_....
00000070: 0000 000d 0000 000e 0000 000f 0000 0010 ................
00000080: 0000 0011 0001 9019 0000 0012 0001 fff2 ................
00000090: 0000 0013 0001 fff3 0000 0014 0000 0015 ................
000000a0: 0000 0016 0000 0017 0000 0018 0000 0019 ................
000000b0: 0000 001a 0000 001b 0000 001c 0000 001d ................
000000c0: 0000 001e 0000 001f 0000 0020 0000 0021 ........... ...!
000000d0: 0000 0022 0000 0023 0000 0024 0000 0025 ..."...#...$...%
000000e0: 0000 0026 0000 0027 0001 fff4 0000 0028 ...&...'.......(
000000f0: 0000 0029 0000 002a 0000 002b 0000 002c ...)...*...+...,
00000100: 0000 002d 0000 002e 0002 1191 0002 1838 ...-...........8
00000110: 0001 5f43 0000 0030 0001 fff6 0000 0031 .._C...0.......1
00000120: 0000 0032 0000 0033 0000 0034 0000 0035 ...2...3...4...5
00000130: 0001 6782 0000 0036 0001 6783 0001 6785 ..g....6..g...g.

The files start with a WDP\0 header (which was not found elsewhere on the internet), and a bunch of 0000 bytes spread across the odd columns, but we can’t really interpret the meaning of these data by just staring the files. We definitely need the help of the code logic of the game.

Preparing for decompilation

As it is commonly known, most logic of Unity games are written in C♯. When compiled to DLL files, C♯ code is rather easy to decompile. However, in environments where .NET runtime is hard to prepare, or where performance is critical, Unity offers an option called Intermediate Language to C++ (IL2CPP) that further compiles Microsoft Intermediate Language (MSIL) into C++ and further into native code. This technique is commonly seen on Unity games running on mobile platforms. Nintendo Switch is of no exception.

Nintendo Switch runs on a special binary format called NSO, which is a custom variant of AArch64 ELF binary. To save space, a lot of NSO files are by default compressed. We need to first decompress it with hactool.

hactool --uncompressed=exefs/main_unc exefs/main

With the uncompressed binary, we can then use IL2CPPdumper to extract the offset and signature of each function in the binary.

Perfare / Il2CppDumper

Unity il2cpp reverse engineer

Il2CppDumper

中文说明请戳这里

Unity il2cpp reverse engineer

Features

Complete DLL restore (except code), can be used to extract MonoBehaviour and MonoScript
Supports ELF, ELF64, Mach-O, PE, NSO and WASM format
Supports Unity 5.3 - 2022.2
Supports generate IDA, Ghidra and Binary Ninja scripts to help them better analyze il2cpp files
Supports generate structures header file
Supports Android memory dumped libil2cpp.so file to bypass protection
Support bypassing simple PE protection

Usage

Run Il2CppDumper.exe and choose the il2cpp executable file and global-metadata.dat file, then enter the information as prompted

The program will then generate all the output files in current working directory

Command-line

Il2CppDumper.exe <executable-file> <global-metadata> <output-directory>

Outputs

DummyDll

Folder, containing all restored dll files

Use dnSpy, ILSpy or other .Net decompiler tools to view

Can be used to extract Unity MonoBehaviour and MonoScript, for UtinyRipper, UABE

ida.py

For IDA

ida_with_struct.py

For IDA, read il2cpp.h file and apply structure…

View on GitHub

Il2CppDumper exefs/main romfs/Data/Managed/Metadata/global-metadata.dat il2cppdump

In the new il2cppdump folder, you can find the JSON file script.json and a C++ header file il2cpp.h with all the metadata, which we will use later to locate the function code during the actual decompilation.

Browsing the script.json file, we can find some interesting methods that might help us to decode the dictionary file:

void CDictionary__GetDictionaryData (CDictionary_o*__this, System_String_o **strReading, System_String_o** strNotation, System_String_o** strMeaning, int32_t nWordId, const MethodInfo* method);
int32_t CDictionary___ConvertKey2String (CDictionary_o*__this, System_String_o** strOut, System_UInt32_array* apKey, uint32_t nLongFlag, const MethodInfo* method);

Fortunately, not only the class and method names, even the parameter names are kept, which will help us a lot figuring out the code logic.

To actually decompile the file, we will use Ghidra, an open-source reverse engineering tool that works on multiple platforms.

NationalSecurityAgency / ghidra

Ghidra is a software reverse engineering (SRE) framework

Ghidra Software Reverse Engineering Framework

Ghidra is a software reverse engineering (SRE) framework created and maintained by the National Security Agency Research Directorate. This framework includes a suite of full-featured, high-end software analysis tools that enable users to analyze compiled code on a variety of platforms including Windows, macOS, and Linux. Capabilities include disassembly assembly, decompilation, graphing, and scripting, along with hundreds of other features. Ghidra supports a wide variety of processor instruction sets and executable formats and can be run in both user-interactive and automated modes. Users may also develop their own Ghidra extension components and/or scripts using Java or Python.

In support of NSA's Cybersecurity mission, Ghidra was built to solve scaling and teaming problems on complex SRE efforts, and to provide a customizable and extensible SRE research platform. NSA has applied Ghidra SRE capabilities to a variety of problems that involve analyzing malicious code and generating deep…

View on GitHub

However, Ghidra does not support NSO and IL2CPP binary out of the box, so we need some install something more to help us, namely:

Ghidra Switch Loader, which can be installed by going to File -> Install Extensions… in Ghidra and click the + button at the corner.
Adubbz / Ghidra-Switch-Loader

Nintendo Switch loader for Ghidra
Ghidra Switch Loader

A loader for Ghidra supporting a variety of Nintendo Switch file formats.

Building
- Ensure you have JAVA_HOME set to the path of your JDK 17 installation.
- Set GHIDRA_INSTALL_DIR to your Ghidra install directory. This can be done in one of the following ways:
  
  Windows: Running set GHIDRA_INSTALL_DIR=<Absolute path to Ghidra without quotations>
  
  macos/Linux: Running export GHIDRA_INSTALL_DIR=<Absolute path to Ghidra>
  
  Using -PGHIDRA_INSTALL_DIR=<Absolute path to Ghidra> when running ./gradlew
  
  Adding GHIDRA_INSTALL_DIR to your Windows environment variables.
- Run ./gradlew
- You'll find the output zip file inside /dist
Installation
- Start Ghidra and use the "Install Extensions" dialog (File -> Install Extensions...).
- Press the + button in the upper right corner.
- Select the zip file in the file browser, then restart Ghidra.
View on GitHub
ghidra.py from IL2CPPdumper, which can be installed by copying the file to the %USERPROFILE%/ghidra_scripts folder.

Decompile the binary

Finally, we can proceed to decompile the binary. To make full use of the metadata we extracted earlier, there are a few steps we need do before starting to read the source code.

When the main_unc bianry is first loaded into a Ghidra project, it will prompt you to start an automatic analysis. Since the binary contains about 47MB worth of data, it might take a considerable amount of time to conduct the analysis, and we are only interested in a small portion of the code. I thus chose to skip the analysis.

The first step is to import the data types defined in the header file into Ghidra. Since the generated il2cpp.h contains some data types that it does not recognize natively, we need to prepend these lines to it.


typedef unsigned __int8 uint8_t;
typedef unsigned __int16 uint16_t;
typedef unsigned __int32 uint32_t;
typedef unsigned __int64 uint64_t;
typedef __int8 int8_t;
typedef __int16 int16_t;
typedef __int32 int32_t;
typedef __int64 int64_t;
typedef __int64 size_t;
typedef size_t intptr_t;
typedef size_t uintptr_t;

With the modified header file, we can then return to Ghidra, open File -> Parse C Source… to import it. When the Parse C Source dialog is opened, clear everything in the Source files to parse and Parse options section, then add the header file we prepared. Finally, click Parse to Program to start.

Clear everything and add only the il2cpp.h file to the list.

Next, we need to label the functions at their respective offsets. Open the Script Manager from Windows -> Script Manager, search for ghidra.py, then click the green play ⃝▶ button to run the script. When prompted for files, select the script.json file exported from IL2CPPdumper.

Once the script is finished, we will see there will be all the functions imported in the Symbol Tree panel in the sidebar. In the Filter box of at the bottom of the section, we can enter CDictionary to find all the dictionary related method.

Symbol Tree panel with the imported functions

Then we can select the CDictionary$$GetDictionaryData in the Symbol Tree, and Ghidra will direct us to the correct byte offset in the Listing window. If you see a bunch of ?? in the listing, press D on the keyboard to disassemble the function. As it disassembles, some C-like source code will also show up on the right side in the Decompile window.

Inside the window, you may see a lot of types are set as long, unknown8, and strange type that might not make sense. A lot of these can be fixed by correcting the function signature and let Ghidra to re-infer the types. To do so, right click the function name and click Edit Function Signature. In the dialog opened, replace the arguments and return type in the large text box with what you can find in the corresponding "signature" field in the script.json file. Take note that you need to drop the MethodInfo * method argument (which is always the last argument), as it is not decompiled by Ghidra.

To help improving the readability of the decompiled source, here are some helpful shortcut keys:

; to add comments
L to rename variable
Ctrl + L to reassign type of a variable
Mouse middle click to highlight all reference of the variable

With correct types, Ghidra is able to better infer some pointer offsets as struct properties or array indexes, which makes. the decompiled source easier to read.

Disassembled listing and annotated decompiled source code side by side

Debugging with emulator

With all the source disassembled and decompiled, it is sufficient to conduct a static analysis and recover most of the logic to parse the dictionary. However, there are still some portion of the code where the decompiled code does not make sense potentially due to some misassigned types.

int CDictionary$$_ConvertKey2String
              (CDictionary_o *__this,System_String_o **strOut,System_UInt32_array *apKey,
              uint32_t nLongFlag)

{
  undefined8 ex;
  CMojiBlock_o *mojiBlock;
  System_String_o *result;
  int i;
  ulong apKey10;
  ulong uVar1;
  long lVar2;
  float *matrix03;
  uint apKey0;
  uint apKeySize;

  if ((DAT_7102d7e960 & 1) == 0) {
    __this = (CDictionary_o *)InitializeMethodMetadata(0xc24);
    DAT_7102d7e960 = 1;
    apKeySize = *(uint *)&apKey->max_length;
  }
  else {
    apKeySize = *(uint *)&apKey->max_length;
  }
  if (apKeySize == 0) {
    ex = IndexOutOfRange_FUN_7100bd84b0(__this);
    __this = (CDictionary_o *)throw_FUN_7100bd72c0(ex,0,0);
    apKeySize = *(uint *)&apKey->max_length;
    apKey0 = apKey->m_Items[0];
  }
  else {
    apKey0 = apKey->m_Items[0];
  }
  if (apKeySize < 2) {
    ex = IndexOutOfRange_FUN_7100bd84b0(__this);
    throw_FUN_7100bd72c0(ex,0,0);
  }
  result = EmptyString;
  apKey10 = (ulong)apKey0 & 0x7fffffff;
  if (nLongFlag != 0) {
    apKey10 = CONCAT44(apKey->m_Items[1],apKey0);
  }
  i = 0;
  uVar1 = (ulong)((uint)apKey10 & 0x7f);
  *strOut = EmptyString;
  if ((apKey10 & 0x7f) != 0) {
    do {
      if (((*(byte *)(SingletonMonoBehaviour<GlobalFunc>_TypeInfo + 0x127) >> 1 & 1) != 0) &&
         (*(int *)(SingletonMonoBehaviour<GlobalFunc>_TypeInfo + 0xd8) == 0) ) {
        ExclusiveMonitor_FUN_7100bb3c20();
      }
      mojiBlock = (CMojiBlock_o *)
                  SingletonMonoBehaviour<_CMojiBlock>$$get_Instance
                            (Method$SingletonMonoBehaviour<GlobalFunc>.get_Instance() );
      matrix03 = *(float **)&(mojiBlock->fields).m_MojiMtx.fields.m03;
      lVar2 = (long)(int)uVar1 + -1;
      if ((uint)matrix03[6] <= (uint)(float)lVar2) {
        ex = IndexOutOfRange_FUN_7100bd84b0();
        throw_FUN_7100bd72c0(ex,0,0);
      }
      /* What is this mess trying to get a string from a Unity matrix offset? */
      result = System.String$$Concat(result,*(System_String_o **)(matrix03 + lVar2 * 2 + 8) );
      uVar1 = apKey10 >> 7 & 0x7f;
      apKey10 = apKey10 >> 7;
      i = i + 1;
      *strOut = result;
    } while ((int)uVar1 != 0);
  }
  return i;
}

Without much clue to untangle this mess, I thought it would be easier to get the game running and attach a debugger to it to actually see how it works. Luckily there’s Yuzu, a Nintendo Switch emulator that comes with GDB stub that can allow us to attach a GDB session to it.

yuzu-emu / yuzu

Nintendo Switch emulator

yuzu

yuzu is the world's most popular, open-source, Nintendo Switch emulator — started by the creators of Citra
It is written in C++ with portability in mind, and we actively maintain builds for Windows, Linux and Android

Compatibility

The emulator is capable of running most commercial games at full speed, provided you meet the necessary hardware requirements.

For a full list of games yuzu supports, please visit our Compatibility page.

Check out our website for the latest news on exciting features, monthly progress reports, and more!

Development

Most of the development happens on GitHub. It's also where our central repository is hosted. For development discussion, please join us on Discord.

If you want to contribute, please take a look at the Contributor's Guide and Developer Information You can also contact any of the developers…

View on GitHub

To enable GDB on Yuzu, go to Emulation -> Configure…. In the popup, go to General -> Debug -> Debug, and check Enable GDB Stub. To the right end of the check box is the port number where Yuzu is listening for GDB connections.

GDB Stub settings in Yuzu

Once GDB Stub is enabled, the game will only initialize the essential parts, and pause to wait for us to inspect it, and set up breakpoints before we ask it to continue.

While Yuzu conveniently provides a way for us to plug GDB into the emulator, not all GDB would work with it. As NSO binaries are essentially AArch64 binaries, we need a GDB that’s compiled to support this architecture to work with it. Fortunately, devKitPro has offered a GDB that’s compatible with AArch64. Once devKitPro is installed, run the following command to install GDB for AArch64:

dkp-pacman -Syu devkitA64-gdb

With this, we are ready start debugging with GDB and Ghidra on Yuzu. In the Ghidra project window, click the 🪲 icon in the toolbar to open the Ghidra debugger.

In the debugger window, look for the Debugger Targets panel to the left, and click the Create a new connection to an (sic.) debugging agent. In the dialog, select IN-VM GNU gdb local debugger, and enter the full absolute path of the previously installed gdb for AArch64 in the GDB launch command field.

Screenshot of the Ghidra Connect dialog to specify the GDB command

Once finished, there will be a new Interpreter panel shown up to the right, with a (gdb) prompt at the bottom of the panel. To connect the GDB session to Yuzu, use the following command:

(gdb) target extended-remote 127.0.0.1:5678

…where 5678 is the port number previously set in Yuzu settings.

If the connection is successful, you can try to inspect the offset of the running game by running monitor get info. You should get an output similar to this:

(gdb) monitor get info
Process: 0x51 (main)
Program Id: 0x01006b900f436000
Layout:
  Alias: 0x108c600000 - 0x208c600000
  Heap: 0x208c600000 - 0x220c600000
  Aslr: 0x0008000000 - 0x8000000000
  Stack: 0x100c600000 - 0x108c600000
Modules:
  0x0008000000 - 0x0008003fff rtld
  0x0008004000 - 0x000b091fff main
  0x000b092000 - 0x000b7a5fff subsdk0
  0x000b7a6000 - 0x000c4a2fff sdk

In the output, we can see a line that says 0x0008004000 - 0x000b091fff main. This tells you the address range of our binary is mapped in the RAM. In this case, the starting address in the binary 0x710000000 is corresponding to 0x0008004000 in the RAM from the debugger.

To let Ghidra to match the RAM data against our disassembled binary, we can first drag the main_unc item from the Ghidra project window to the debugger, which allows us to see the RAM view and the decompiled listing side by side.

Then, in the Module tab of the right sidebar of the debugger, click the 📄 icon to open the Static Mapping dialog. In the dialog, press the  button to create a mapping. In the Add Static Mapping dialog, fill the following fields accordingly:

Static range : [ram: starting address in listing, ]
Dynamic range : [ram: starting address of the main module,ending address of the main module]
Length : (auto filled)
Lifespan : (-∞ .. +∞)

Add Static Mappings dialog set with the instruction above

Click Apply to save.

Now, we have the disassembled and annotated binary mapped against the memory space. We can easily navigate like we did in the CodeBrowser. Functions are listed in the Symbol browser at the same place, and we can navigate back to CDictionary$$GetDictionaryData from there.

To set a breakpoint, select the instruction from the Dynamic, Listing, or Decompile panel, and press K.

Once all necessary breakpoints are set, we can move back to the Interpreter window, and type continue to resume the game.

While debugging, some important tools and informations can be found in different panels of the debugger window:

Step into, Step over, Continue are available in the Object panel to the left.
Call stack is available at the Stack panel at bottom left.
Register values are available at the Registers panel to the right. Right click a register can also jump to the Dynamic view to the memory address.

All GDB commands are still available from the Interpreter panel_._

To trigger the game to load up from the dictionary with predictable result, the easiest way is to search for a word in the game’s word lookup feature. Once we hit submit, the game is paused for us at the beginning of the GetDictionaryData method as the first breakpoint hit.

Screenshot of the word lookup feature in the game

With help of these tools and the memory value at each step, it is much easier to understand the logic of the code, and to untangle those sections that the decompiler did not handle properly.

By inspecting the memory content around the matrix offset, It turns out that that part of the code is actually loading from a list of strings that converts the key into a Hiragana, which happened to follow the Unicode order with a few exceptions.

From there, I was able to fully decode the dictionary files and extract all words with their definitions, which wraps up the project with a success.

I have released the parsing script, and the parsed dictionary data as JSON file to a GitHub repository. There are a lot of binary processing due to the nature of the format, but is in general much more readable. Note that thre are still other data not used in the dictionary files, but since we are only interested in the words and definitions, I think I have achieved the goal.

blueset / MojipittanDictionary

Extracted dictionary data from Kotoba no Puzzle: Mojipittan Encore.

Mojipittan Dictionary

Extracted dictionary data from Kotoba no Puzzle: Mojipittan Encore (『ことばのパズル　もじぴったんアンコール』).

Files

worddata.aid, worddata.cot, worddata.dic – original data from game
decoder.py – decoder for the files
dictionary.json – extracted dictionary data

View on GitHub

Trivia
As it is obvious from the game’s word lookup feature, the longest word in the dictionary has 9 kana, which means リバース・エンジニアリング (reverse engineering) in the cover picture doesn’t actually exist in the dictionary, although リバース and エンジニアリング do. Also, ユニティー (Unity) isn’t in the dictionary either despite being short enough.

Fun fact
In order to save space, Mojipittan Encore uses Shift JIS instead of UTF-8 to store the phrases and definitions, as most characters used take 2 bytes in Shift JIS against 3 in UTF-8. However, this has a problem where some accented letters, like the é in café and cliché, are not encoded in Shift JIS. How the game solves it is to encode these characters as question marks in the dictionary, and hardcoded all the affected words in the game logic when converting them back to UTF-8.

The post Reverse engineering an IL2CPP NSO binary: Case study of Mojipittan Encore appeared first on 1A23 Blog.

Use WebVTT without actually using WebVTT: Another way to monitor playback progress of HTML Media Elements

Eana Hufwe — Sun, 15 Jan 2023 20:37:25 +0000

Previously, I have introduced how LyricsX handled playback progress of different players, and briefly talked about how I applied its principal to web audio with a requestAnimationFrame() loop. In this article, I’ll talk about how to use WebVTT, a browser-native captioning feature to receive callbacks on specific time ranges.

Web Video Text Tracks (WebVTT) is a media captioning feature of HTML5 standard that has support from all major modern browsers. In its typical use cases, author of the webpage can provide a caption file in an SRT-like syntax defined by WebVTT, and link the file to a video element using a <track /> tag. The browser will then be aware of the caption, and provide option to render it on top of the video.

Besides rendering captions, WebVTT also provides a set of API that accepts a callback function when the media element enters a specific time range, which we can make use of to track the playback progress without using the sometimes-not-as-precise timeupdate event. While WebVTT is designed for video playbacks, the callback features would also work even the track is attached to an <audio /> node.

In WebVTT, each caption line with a specific time range is called a “cue”. To provide callback functions to a cue, the cue has to be inserted to a caption track (also known as a textTrack) programmatically.

To add a track, it is recommended to create a <track /> node inside the audio/video node. Despite the Multimedia API offers a .addTextTrack() method, tracks added in this way cannot have an ID, and cannot be removed until the page refreshes due to the lack of a corresponding .removeTextTrack() method. Therefore, using DOM nodes is safer in terms of controllability.

Below is an example of creating a text track programmatically.

const track = document.createElement("track");
const uniqueId = Math.random().toString(36).substring(2, 15);
track.id = `track-${uniqueId}`;
track.kind = "subtitles";
track.label = `Track #${uniqueId}`;
// Minimum WebVTT track file
track.src = "data:text/vtt;base64,V0VCVlRUCgoK";

const player = document.querySelector("audio");
player.appendChild(track);
// Enable track in order to allow callbacks
track.track.mode = "hidden";

In case you need to remove the track to start over, it is as easy as removing the track element from the dom.

track.parentNode.removeChild(track);

WebVTT tracks provides convenient API to add and remove tracks. When adding a track, it needs 3 parameters, start time and end time in seconds, and the subtitle content. In our use case, the subtitle content is optional, and you can leave it blank, or set it to any content you prefer for debugging purpose.

const cue = new VTTCue(1.5, 2.25, "");

Then, you can add event listeners to the cue object you created. VTTCue has two events, enter and exit. As the name suggest, these events are triggered when the current playback progress enters or exits the cue’s time range. It will trigger in normal playback, and also when user seeks the progress bar.

For use cases where the events are single-ended, i.e. events only has a start time, and ends right before the next starts, it is possible to only set an enter event listener, and let the enter event of another cue as the exit event of the current one.

cue.addEventListener("enter", () => {
    console.log("Entered cue");
});

Last but not least, the cue need to be added to a track to take effect.

track.track.addCue(cue);

Now, when you play the media element with the track attached, you should be able to see the events triggered as it enters the time duration of each cue.

While this WebVTT implementation is more native to the browser and technically offering better performance than listening to timeupdate and scan through all time ranges every time, the precision of this is still being limited by the browser’s WebVTT implementation itself. On Chrom* browsers, it does not guarantee a perfect sync with the media, which is also observed with ordinary WebVTT subtitles.

If you need a better precision control, and only work with short audio files, Web Audio API might be a better choice.

If you are interested in a full example, and you are comfortable with TypeScript and React, here’s an exmaple where I apply this WebVTT track-based tracking to a custom React hook in Lyricova Jukebox: hooks.ts.

The post Use WebVTT without actually using WebVTT: Another way to monitor playback progress of HTML Media Elements appeared first on 1A23 Blog.

Flexible and dynamic flow control of Azure DevOps YAML Pipelines using variables

Eana Hufwe — Sun, 15 Jan 2023 03:40:52 +0000

Recently I was working on the release automation at work, and one of the requirements is to gap a specific number of hours between stages, and snap to normal business hours. While having a centralized scheduler won’t be a choice unless I want to flood the run logs, and a pipeline run would mostly be unconfigurable once started due to the constraint of Azure DevOps (AzDO) Pipelines, there are still some trickeries to achieve dynamic flow control within the pipeline. In this article, I’d talk about how I setup the flow control.

Here are the two requirements that we want to address in this article:

To leave a gap of an arbitrary duration (computed at runtime) between two stages.
To cut off a branch of the pipeline run, whose condition is determined at runtime, without leaving a failure status, while respecting other settings of the run like skipped stages and cancellations.

Both of them can be achieved on the same YAML Pipeline.

Arbitrary gap between stages

While AzDO YAML pipelines does not come with a true delay-between-stages feature similar to the one in Classic Release Pipelines, inserting a delay within in a YAML pipelines Job is rather simple. AzDO Pipeline comes with a Delay task that can wait for up to 60 days. We’ve tested it on our pipeline that it can wait for at least 72 hours without issue.

There are two things need to be taken care of. One is that the Delay task is an agentless task. If most of your tasks run on an agent, like PowerShell or Bash script, you would need to create a separate _agentless job _ for the Delay task. The other is that AzDO Pipelines by default limit a job/task timeout to be 60 minutes. If you expect it to delay for longer than that, you should update the cancelTimeoutInMinutes of both the task and the containing job to the maximum duration you’d expect, or simply put 0 to eliminate the limit.

To set a duration dynamically, we can calculate the duration in a job, and set the result with an output variable. In agent tasks, you can set an output variable by printing a logging command to stdout. You can also invoke the REST API of AzDO Pipelines if you are running a custom agentless task to achieve similar outcome.

Below is an example demonstrating a way to schedule a gap for at least 48 hours, and the gap will always ends on a weekday.

- job: GapScheduler
  displayName: Gap scheduler
  steps:
  - name: Scheduler
    powershell: |
      $delayDuration = New-TimeSpan -Hours 48
      while (
        (Get-Date).Add($delayDuration).DayOfWeek -eq 'Saturday' -or 
        (Get-Date).Add($delayDuration).DayOfWeek -eq 'Sunday'
      ) {
        $delayDuration = $delayDuration.Add(New-TimeSpan -Hours 24)
      }
      Write-Host "##vso[task.setvariable variable=delayMinutes;isoutput=true]$($delayDuration.TotalMinutes)"
- job: InterStageGap
  displayName: Inter-stage gap
  dependsOn: GapScheduler
  pool: server
  cancelTimeoutInMinutes: 5760 # 4 days
  steps:
  - task: Delay@1
    inputs:
      delayForMinutes: $[dependencies.GapScheduler.outputs['Scheduler.delayMinutes'] ]

Cut off a pipeline branch

By cutting off a pipeline branch, I mean that when a certain condition is met, all stages depending on it, both directly and indirectly, should be skipped without leaving an error state. Other stages not on the dependency chain shall not be affected. Also, this shall not affect stages that has their dependency disabled at trigger time. It shall also not affect existing behavior when there are legitimate failures.

To decide if a stage should be ran, the condition should be decided before it starts running, that is, in its dependency stage. The decision can be made in any sort of task that can set an output variable. Since the output would be a boolean-like value, and the only type you can set for a variable is string, you can put whatever value you like as the true/false value, here we will use "true" and "false" for simplicity.

To skip a stage based on an output of a previous stage, we need to use the stage condition property. Apart from the existing default condition, we also need to check if the previous stage has told us to skip.

There could be three kinds of “output” from the previous stage decision, "true", "false", and null which could happen if the previous stage is disabled at trigger time. Since we want the stage to run in both the "true" case and the null case, we only need to check if the variable is set to "false".

The default condition for a stage is succeeded(), which has covered all existing conditions that can properly handle failure and disabled cases. It is also tested that succeeded() evaluates to false when the previous stage is skipped by a condition expression that was evaluated to false. In this way, we can safely extend from there with an and() clause.

Below is an example that skips Stage2, and subsequently Stage3 on every Monday.

- stage: Stage1
  jobs:
  # Other jobs go here...
  - job: CutOffDecider
    displayName: Cut-off decider
    steps:
    - name: Decider
      powershell: |
        $decision = "true"
        if (Get-Date.DayOfWeek -eq "Monday") {
          $decision = "false"
        }
        Write-Host "##vso[task.setvariable variable=shouldRunNextStage;isoutput=true]$decision"        
- stage: Stage2
  dependsOn: Stage1
  condition: and(succeeded(), ne(dependencies.Stage1.outputs['CutOffDecider.Decider.shouldRunNextStage'], 'false'))
  jobs:
  # Jobs go here...
- stage: Stage3
  dependsOn: Stage2
  # Stage 3 is also skipped as a part of the dependency tree when Stage 2 is skipped
  jobs:
  # Jobs go here...

Dynamic gap and cut-off control together forms two of the building blocks of the foundation of a flexible and dynamic multi-stage release automation system that works on Azure DevOps YAML pipelines.

The post Flexible and dynamic flow control of Azure DevOps YAML Pipelines using variables appeared first on 1A23 Blog.

Content-aware Infinite Scroll Loop using JavaScript

Eana Hufwe — Mon, 11 Jul 2022 02:52:58 +0000

This project came out from a key highlight of the Luna for CTFd theme, a CTFd theme trying to reproduce the atmosphere of the game Project SEKAI: Colorful Stage feat. Hatsune Miku. In order to recreate the unique and symbolic music selection interface, I went forward to write this piece of code out myself.

Below is a simple demo of the final outcome.

Before I started coding, the first thing I did was to look for existing implementations of scroll loops. There are mainly two types of implementation, one is to have the scroll content overflow the container, and programmatically manipulate the scroll position, the other is to append elements dynamically to the end of the scroll content when user scrolling near to the end.

I went with the former strategy because the latter would only allow scrolling in a single direction. However, the former strategy relies on the fact that the list stretching beyond the view size, and has no consideration on the selected item within the list.

Here is a list of features I implemented for the scroll loop list, and I will talk about them one by one in detail afterwards.

Repeat the list items at least once both above and below the original list to cover the entire screen. This is to ensure that we have enough content for the loop transition to happen smoothly.
Scroll in both directions should jump forward or backwards accordingly to ensure a smooth loop.
- When an item is selected, once it is being scrolled out of the screen, it should appear again in the opposite side following the scroll direction once appeared.
When an item is selected, it should be highlighted , and there should be a smooth transition to move it to the center of the screen.
The list should able to be updated when filtered with different criteria , and the selected item will be cleared when the criteria changes.

Repeating the list items

Repeating the list items is seemingly the easiest problem to solve here. The only factor we need to determine the number of times we need to repeat each side. Usually this factor is determined by the size of the viewport, and must be updated when the window resizes. However, in this example, we are going to use the screen size as the basis for performance reasons. In this way, we do not need to keep track of the size of window constantly.

Here we assume that the screen size is constant, that means the script cannot handle cases where the window is being dragged to another screen with a larger size, or is rotated 90°. Yet these the size will be recalculated when the user switches to another category, so it is not too much of an issue.

The number of times we need to repeat the list can be found with an easy division:

\left\lceil{\text{height of screen}\over\text{height of an item}\times\text{number of items}}\right\rceil

Hereafter, we will refer to the list at the center as the “center segment” of the list.

Scroll jumps to the opposite direction for a smooth loop

This is the most basic requirement to achieve a scroll loop. When the user scrolls to the top or the bottom end of the list, we let the browser jump back to the opposite end to allow the user to continue scrolling. This is done by listening to the scroll event of the scrolling container and update the scroll offset when necessary.

Below is an animation illustrating the logic.

Watch the animation on my blog.

As illustrated, despite there is a sudden jump in the scroll position, visually the user would feel like the list is kept on scrolling because the content in the viewport remains the same before and after the jump.

The remaining question would be when we jump and how far we jump. I will talk about these case by case.

Design decision: Item selection

The original game has taken an approach to always select the centered item while scrolling. While this approach may eliminate the need of differentiate the jump logic, in my use case, each selection of an item would send an HTTP request to the server. Selecting items while scrolling would unnecessarily increase the load of the server, and the strategy is thus abandoned.

Scenario 1: No items are selected

When no items are selected, all of the repeating segments will be fully identical to the center one. In this case, we will use the center of scrolling container as a basis. When the center segment is scrolled pass the basis, we jump the scroll position for a full height of the center segment to ensure that the basis line always overlap with the center segment.

The length of the jump must always be a multiple of the center segment height to ensure that we do not create any visual disruption while manipulating the scroll position.

Scenario 2: An item is selected

Once an item is selected, the center segment will have an item highlighted comparing to the repeating segments. With the highlighted row, users would tend to use it as a visual probe when scrolling through the list. This means we can no longer jump through the list bluntly, as for shorter lists the selected item will be jumping around in the middle of the screen, which is less ideal visually.

Instead, we want the select item loop across the screen like the Snake, while still keeping it always within the scroll window whenever possible.

The logic I had for this scenario can be summarized in the pseudo code below:

if the selected item is out of view:
  if the same item in the previous or next repeating segment is in the view:
    // Jump to the furthest selected item in the view.
    jump max(⌈viewHeight ÷ centerHeight⌉, 1) × centerHeight
  else:
    // The view is in between two selected item
    Follow the logic in Scenario 1

The reason of not highlighting the item in every repeating segment is because that the view will be visually too busy when multiple segments are shown in the view. We only want to keep the selected item in the center segment.

Below is an animation illustrating an example of this logic, where ★ signifies the selected item.

Watch the animation on my blog.

Smooth transition to the center when an item is selected

With modern browser, smooth scroll to a specified offset is no longer hard, the scrollTo API offers an option behavior: "smooth". Specifying the offset, and the browser will do the transition animation for you. The only one trick we would need do is to ensure that the item clicked is from the center segment for highlighting.

The logic is rather simple, first check if the item clicked is within the center segment. If not, jump the scroll offset by the distance between the clicked item and the corresponding item in the center segment. Finally do a scroll transition.

This transition gives users a positive feedback that the web app has responded to their selection.

Updating the list with filters

Updating the list is in fact mostly outside of the scope of the scroll loop project, expect that the number of repeat times need to be recalculated when the length of filtered items changes.

To achieve this, you can choose whichever framework you like, or even vanilla JavaScript. In the example above, I used Alpine.js which is the front end framework used by the default CTFd theme.

There is one thing you may need to take note when using Alpine.js: clearing selection must be done after populating the filtered list. Otherwise the repeated list might not be updated correctly due to a rendering disruption.

Browser Compatibility

While this solution works for most modern browsers, there is a specific problem with Firefox (Gecko) for Windows. As outlined in their Wiki article about scroll-linked effects, scrolling events in Firefox for Windows are sent asynchronously, and usually comes with a delay. That means the user scroll will sometimes override the value set by JavaScript, causing the screen to jitter as a result.

While the article has proposed various JavaScript API for use cases that need to manipulate the scroll position itself, none of them is implemented at the moment. With this, I had to resort to a suboptimal solution, debounce the scroll position update event calls until the user stops scrolling. While this may make the scroll manipulation seem slow, it is still better than constantly jittering user’s screen during scrolling.

For reference, Firefox for Android, macOS and Linux works just fine as Chrome and Safari do.

While some of the solutions may sound hacky and not perfect across all forms, this is the best I can come up with at the moment. I definitely look forward to more advanced scroll manipulation API being implemented for a simpler and better solution that this.

Feel free to leave a comment if you have any questions or suggestions. Hope you enjoyed this!

The post Content-aware Infinite Scroll Loop using JavaScript appeared first on 1A23 Blog.

Extract and Visualize Your Telegram Group Network

Eana Hufwe — Fri, 02 Jul 2021 04:52:04 +0000

This was one of the ideas I wanted to implement for a while: to visualize the members of all groups I joined on Telegram, and see how many circles I had on the single platform. Thanks to the openness of Telegram’s client API, this was rather easy to achieve.

A sneak peek of what I got.

Get prepared

Here is a list of things you’ll need to build your own graph

Python 3
Pyrogram (pip3 install pyrogram)
Networkx (pip3 install networkx)
Telegram account and Client API ID (not to be confused with Bot API Token)
Gephi, Graphistry or other graph visualization tools

Gather data

This time I used Pyrogram instead of Telethon, as I started working on this in a REPL instance in Python, and it used to have a synchronous-flavor that served the purpose best. But I was wrong, they have already got rid of the sync flavor in a previous version. So I chose to turn to iPython for REPL because it can automatically run any async function without the bootstrap code.

Anyway, no worry for you, I’ve already consolidated it to a Python script that handles all the async stuff properly. The code is not complicated in any way, just iterate through all the dialogs and try load through the members whenever possible, and then dump the data into two Pickles files. It should be really easy to rewrite this part with other libraries like Telethon if you already have a session file ready.

Remember to replace the API ID and API hash with your own ones. If you have already logged in with Pyrogram, rename the session name with proper one for Pyrogram to pick up the proper session file you have, otherwise you should be prompted to log in for the first time.

from pyrogram import Client
import pickle
import asyncio

api_id = 123 # your API ID here
api_hash = "1234567890abcdef1234567890abcdef" # your API hash here
c = Client("my_pyrogram_session", api_id, api_hash)

async def main():
    await c.start()

    dialogs = [i async for i in c.iter_dialogs()]

    chat_members = {}
    for i in dialogs:
        print("Loading", i.chat.id, i.chat.first_name or i.chat.title)
        try:
            members = [j async for j in i.chat.iter_members()]
            chat_members[i.chat.id] = members
            print(len(members), "member(s) loaded.")
        except Exception as e:
            print(e)

    with open("dialogs.pkl", "wb") as f:
        pickle.dump(dialogs, f)

    with open("members.pkl", "wb") as f:
        pickle.dump(chat_members, f)

asyncio.run(main())

Construct graph

Now, with the full data collected, we can construct a graph for other tools to visualize. In this example, we will use Networkx to construct the graph data and optimise it for visualization.

You may want to get rid of all nodes that connects to yourself for a better visualization. Get your ID on Telegram with any bot that’s capable of doing it, and put it on line 4 of the following code.

import pickle
import networkx as nx

MY_ID = 12345678

# Load data
dialogs = pickle.load(open("dialogs.pkl", "rb"))
members = pickle.load(open("members.pkl", "rb"))

# Build graph
g = nx.Graph()
for k, v in members.items():
    for m in v:
        if m.user.id != MY_ID:
            g.add_edge(k, m.user.id)

Now we have the graph, we can continue to work on it to make the visualization look better. Since my graph had 10k+ of nodes and 20k+ of edges, it would be very difficult to visualize graphs of this scale. I then removed all nodes that only has one edge, that is, all people that shares only 1 common group with me.

g2 = g.copy()
for k, v in nx.degree(g):
    if v == 1:
        g2.remove_node(k)

With this removal, I now have 8k+ nodes and 18k+ edges in the graph, reducing the size by about 20%.

Visualize with Gephi

Gephi is an open-source tool for graph visualization, and works especially well on large graphs like ours. However, getting graphs into the software last updated in 2017 could be tricky.

Through some experimentations, I found out that DOT format works the best importing to Gephi, so let’s get started.

from networkx.drawing.nx_pydot import write_dot
write_dot(g2, 'graph.dot')

Open the graph.dot file in your working directory with Gephi, and you should be able to see the graph shown in the window. Usually, it will automatically generate a layout for you. If you are not satisfied with it, I’d recommend to reset it with Random Layout and build a new layout again with ForceAtlas 2.

Choose the layout engine you want and then click run. Note that you might need to click Stop manually if a layout engine runs for too long.

When you are satisfied with the graph, you can export it as a PNG, PDF, or SVG files for sharing.

Adding metadata for Gephi

If you think your computer is powerful enough, you can also consider adding labels and other medatada to the graph, so that you can explore it better in Gephi. Here is an example of adding chat names to the graph as labels.

dmap = {i.chat.id: i.chat for i in dialogs}
for i in members.values():
    for j in i:
        if j.user.id not in dmap:
            dmap[j.user.id] = j.user

for i in g2.nodes.keys():
    if i in dmap:
        t = None
        try:
            if dmap[i].is_deleted:
                g2.nodes[i]["label"] = "%DELETED_ACCOUNT%"
                continue
        except:
            pass
        try:
            t = dmap[i].title
            if t:
                g2.nodes[i]["label"] = dmap[i].title
                continue
        except:
            pass
        try:
            if not t:
                if dmap[i].first_name or dmap[i].last_name:
                    t = f"{dmap[i].first_name} {dmap[i].last_name or ''}".strip()
                    g2.nodes[i]["label"] = t
                    continue
        except:
            pass
        if not t:
            print(i, "has no name")
    else:
        print(i, "is not found")

In case there are people with uncommon Unicode character in their names (mainly control characters and combining characters), here is a script to keep only letter characters in the label.

import unicodedata
def normalize_name(s):
    s = unicodedata.normalize("NFKC", s)
    s = list(s)
    for idx, i in enumerate(s):
        cat = unicodedata.category(i)
        if cat[0] in "CMZPS" or cat == "Lm":
            s[idx] = "_"
    return "".join(s)

for i in g2.nodes:
    if "label" in g2.nodes[i]:
        g2.nodes[i]["label"] = normalize_name(g2.nodes[i]["label"])

After that, just re-export the chart in DOT format, and import it with Gephi.

Some interesting analytics

Besides the chart, you can also get some interesting analytics out of the data that you can’t get easily with an official Telegram client, desktop or mobile.

Connected components and their sizes

In an undirected graph, a connected component is an induced subgraph in which any two vertices are connected to each other by paths, and which is connected to no additional vertices in the rest of the graph¹ In this context, it shows you how many parts are your network on Telegram separated with only you but no one else in common. This command shows you how large each of part of your network is.

print([len(c) for c in nx.connected_components(g)])

Most connected nodes

Most connected nodes here are either groups with most members, or people that shares most groups in common with you. Here is a simple code script that shows you the top 100 most connected nodes.

dg = g.degree()
sdg = sorted(dict(dg2).items(), key=lambda a: a[1], reverse=True)
print(sdg[:100])

Pairs with most common neighbours

Through pairs with most common neighbours, you can find out which 2 groups or which 2 people are the most similar. For groups, the most similar pair shares the most number of common users. Vice versa, the most similar users share the most number of common groups.

This script prints the top 100 pairs of nodes with common neighbors.

from itertools import combinations

pair_common = []
for i, j in combinations(members.keys(), 2):
    try:
        inb = set(g.neighbors(i))
        jnb = set(g.neighbors(j))
        if len(inb) < 4 or len(jnb) < 4:
            continue
        res = len(inb & jnb)
        pair_common.append(tuple(i, g2.nodes[i]["label"], j, g2.nodes[i]["label"], res))
    except:
        continue

pair_common.sort(key=lambda a: a[4], reverse=True)

print(pair_common[:100])

Other visualization tools

Apart from Gephi, there is another tool that can visualize graphs of large sizes. Sharzy recommended Graphistry as an alternative tool that renders in a web browser and offers colors. Here’s a sample from Sharzy’s Telegram Channel.

https://t.me/sharzy_talk/1125

Note : To render with Graphistry, you need to upload your graph data to their server, take caution before you use it.

Here is a sample snippet to upload your graph to Graphistry for rendering.

import graphistry
graphistry.store_token_creds_in_memory(False)
graphistry.register(api=3, protocol="https", server="hub.graphistry.com",
                    token="YOUR_TOKEN_GOES_HERE")

graphistry.bind(source='src', destination='dst', node='nodeid', point_title="label").plot(g2)

The post Extract and Visualize Your Telegram Group Network appeared first on 1A23 Blog.

From Wikipedia. ↩

Use GraphQL subscription to show progress of time-consuming operations

Eana Hufwe — Thu, 28 Jan 2021 17:41:28 +0000

GraphQL is feature-rich query language designed to fit majority of needs of generic online applications. The 3 main types of operations in GraphQL are query, mutation and subscription. While queries and mutations are send in HTTP requests, subscriptions are running on WebSocket, which enables it to receive updates in real time. In this article, I would like to talk about using the subscription to report real-time results and/or progresses to time-consuming operations.

Usually, for time-consuming tasks like server-side downloads, or gathering data from multiple remote sources, it is usually favourable to provide a visual feedback to the user about the progress – or even better – partial results. In GraphQL, we can make use of subscription for the server to stream responses while during the operation.

Schema design

Suppose that we have a query that looks like this:

query {
    multiSearch(keyword: String!): [SearchResult!]!
}

We can refactor it as follows for progress reports:

type MultiSearchProgress {
    totalSources: Int!
    completedSources: Int!
    success: Boolean
}

type subscription {
    multiSearchStatus(sessionId: String!): MultiSearchProgress!
}

type query {
    multiSearch(keyword: String!, sessionId: String): [SearchResult!]!
}

So here, we allow the multiSearch query to take in an optional string parameter sessionId. The client generates a strong random string as a session ID, supplied when making the query. In the meantime, the client can subscribe to to the multiSearchStatus channel with the same session ID to get real time progress from the server.

When the server receives the query and starts its computation, it can take the session ID from the query, and report it to the corresponding subscription channel with progresses. In the subscription payload, we included some common data like total and loaded numbers.

We have also included the succeeded: Boolean to indicate the status of query. This property can has 3 states:

null: query in progress
true: succeeded
false: failed

This can also give the client a hint of when the query finishes, as an additional safeguard to the actual query outcome.

More payloads

If you want to include more information like partial results, we can also add that to the progress type:

type MultiSearchProgress {
    totalSources: Int!
    completedSources: Int!
    newResults: [SearchResult!]!
    success: Boolean
}

In this new definition, we can stream any new search results retrieved during the progress. These new results can be displayed to the user in real time during the query. Optionally, the final query result can be used to replace data retrieved in the subscription, to avoid any loss of results in WebSocket.

Points to note

As subscriptions require a WebSocket to set up, and not commonly used in most GraphQL servers, it could be left out in your environment if GraphQL have already been used. In this case, additional effort is needed throughout the connection path (server, forwarder, load balancer, CDN, etc.) in order for WebSocket to work.
The client is recommended to subscribe with the generated ID before the query is sent, so as not to lose any update issued right after the query started.
If you have no authorization in place for subscriptions, all such progress reports will be available to anyone with the correct session ID at the right time. If the data sent through subscriptions are sensitive or private. It is recommended to use a cryptographically strong random string generator on the client to generate session IDs, and apply authentication wherever possible.

Code sample

Here is a small sample of implementation in this concept in TypeScript. This sample makes use of TypeGraphQL, React, Apollo Server and Client.

// Mock search source interface
interface SearchSource {
  search(query: string): Promise<SearchResult[]>;
}

// Generic interface for data with session ID
interface PubSubSessionPayload<T> {
  sessionId: string;
  data: T | null;
}

@Resolver()
export class MultiSearchResolver {

  sources: SearchSource[];

  @Query(returns => [LyricsKitLyricsEntry])
  public async multiSourceSearch(
    @Arg("query") query: string,
    @Arg("sessionId", { nullable: true }) sessionId: string | null,
    @PubSub("MULTI_SOURCE_SEARCH_PROGRESS") publish: Publisher<PubSubSessionPayload<MultiSourceSearchStatus>>,
  ): Promise<SearchResult[]> {

    const outcomes = await Promise.all(this.sources.map(source => source.search(query).then(data => {
      if (sessionId !== null) {
        // send partial results for each source
        publish({sessionId, data});
        return data;
      }
    })));

    // Send end-of-query update
    await publish({ sessionId, data: null });

    // join SearchResult[][] into SearchResult[]
    const results = outcomes.reduce((prev, curr) => [...prev, ...curr], []);

    return results;
  }

  @Subscription(
    () => MultiSourceSearchStatus,
    {
      topics: "MULTI_SOURCE_SEARCH_PROGRESS",
      filter: ({ payload, args }) => args.sessionId === payload.sessionId,
      nullable: true,
    }
  )
  multiSourceSearchProgress(
    @Root() payload: PubSubSessionPayload<MultiSourceSearchStatus>,
    @Arg("sessionId") sessionId: string,
  ): MultiSourceSearchStatus | null {
    return payload.data;
  }
}

const MULTI_ENGINE_SEARCH = gql`
  query($query: String!, $sessionId: String) {
    multiSourceSearch(query: $query, sessionId: $sessionId) {
      // fields go here
    }
  }
`;

const MULTI_ENGINE_SEARCH_PROGRESS_SUBSCRIPTION = gql`
  subscription ($sessionId: String!) {
    multiSourceSearchProgress(sessionId: $sessionId) {
      // fields go here
    }
  }
`;

const apolloClient = useApolloClient();
const [searchResults, setSearchResults] = useState([]);
const [query, setQuery] = useState([]);

function search() {
  const sessionId = getSecureRandomString();
  setSearchResults([]);

  const subscription = apolloClient.subscribe<{ multiSourceSearchProgress: SearchResult }>({
    query: MULTI_ENGINE_SEARCH_PROGRESS_SUBSCRIPTION,
    variables: { sessionId }
  });

  const zenSubscription = subscription.subscribe({
    next(x) {
      if (x.data.multiSourceSearchProgress !== null) {
        setSearchResults((results) => [...results, x.data.multiSourceSearchProgress]);
      }
    },
    error(err) {
      console.error(`Finished with error: ${err}`);
    },
    complete() {
      console.log("Finished");
    }
  });

  const query = apolloClient.query<{ multiSourceSearch: SearchResult[] }>({
    query: MULTI_ENGINE_SEARCH,
    variables: { query, sessionId },
  });

  const result = await query;

  if (result.data) {
    setSearchResults(result.data.multiSourceSearch);
  }

  zenSubscription.unsubscribe();
}

The post Use GraphQL subscription to show progress of time-consuming operations appeared first on 1A23 Blog.

Measure per-letter dimension of text in JavaScript

Eana Hufwe — Sat, 09 Jan 2021 05:14:49 +0000

TL;DR: Create a Range, set proper start and end points up to the text node with proper offset, then use Range.getBoundingClientRect() to get the dimensions.

As a part of Lyricova Jukebox, we wanted to support inline karaoke swipe animation. With the time tags in the data, it is easy to figure out the time when the animation must reach a certain character. Then we need to figure out a way measure per character dimension for the animation to work.

Initially, I used to do in the way BalanceText does, modifying DOM content and CSS properties to measure the DOM node itself. BalanceText is a JavaScript library I recently read on, it inserts <br> tags in to text in order to achieve a balanced line length by overriding the DOM HTML source of the targeted node directly.

This indeed could work in a way, especially for BalancedText where it operates based on the “break opportunities” within the pure HTML source. However, that brings a problem to some applications that relies on those DOM elements. Unlike BalanceText, we don’t necessary need to modify the content, especially the DOM structure, just to measure its dimensions. The way BalanceText does discards the old DOM subtree, and replaces them with a new one by directly modifying the innerHTML property of the subtree. This means any event listeners and other references to the element are all unusable after the measurement. This is certainly something that we don’t want.

The Range API

After some digging through the internet, I found this Range API of the web, which can select a region of the webpage and measure its dimension. This API is used to represent selections by users in a webpage, but it isn’t necessary to modify the selection in order to use it.

In this way, we can pretty much say that we can measure the per-character width of text on the webpage without any modification to the DOM tree or even the UI.

I will demonstrate this feature by measuring an accumulating width of the text per character, i.e. the width of text from the beginning to the end of the n_th character for all _n.

The general idea is to:

Locate the starting point. In this example we want to measure from the beginning of a node, so we can just use the Range. __proto__.setStartBefore(el) method.
Recursively locate all text nodes in the sub-DOM tree.
For each text node found
1. Set the range to end on each character of the node by using Range. __proto__.setEnd(el, count).
2. Get the width of the range with Range. __proto__.getBoundingClientRect().

Below is a sample code for the process in TypeScript.

/**
 * A generator that recursively find all text nodes in a subtree rooted at el.
 */
function* recursivelyFindTextNode(el: Node): Generator<Node> {
  if (el.nodeType === Node.TEXT_NODE) {
    yield el;
  } else if (el.nodeType === Node.ELEMENT_NODE) {
    for (let i = 0; i < el.childNodes.length; i++) {
      yield * recursivelyFindTextNode(el.childNodes[i]);
    }
  }
}

const result: number[] = [];
const range = document.createRange();
range.setStartBefore(el);

// expand all text nodes found.
const nodes = [...recursivelyFindTextNode(el)];

for (const textNode of nodes) {
const textLength = [...textNode.textContent].length;
  for (let i = 0; i < textLength; i++) {
    range.setEnd(textNode, i);
    const rect = range.getBoundingClientRect();
    if (rect.width !== 0) result.push(rect.width);
  }
}

// Get the total length of the text
range.setEndAfter(nodes.length > 0 ? nodes[nodes.length - 1] : el);
const rect = range.getBoundingClientRect();
result.push(rect.width);

Caveat

Since this API is measuring things rendered on screen directly, if you want to measure something, it has to be rendered first.

Also, if you want to measure the length of a long string of text in pixels, the text must be rendered in one line. This is achievable by a few lines of CSS:

white-space: pre;
display: inline;
width: fit-content;

Finally, here is a simple CodePen demo:

The post Measure per-letter dimension of text in JavaScript appeared first on 1A23 Blog.

How LyricsX keeps track of progress of media players

Eana Hufwe — Wed, 06 Jan 2021 05:50:57 +0000

LyricsX is an open source software for macOS to download and display lyrics of current playing track on Music (previously iTunes), Spotify, Audirvana, Vox, Swinsian, or the Now Playing indicator in the OS. It gets time-tagged lyrics files from local storage or internet, and then display the lyrics in sync with the player.

As a crucial component of the development of Lyricova Jukebox, I have researched multiple implementations of real time lyrics display programs, and I found the mechanism behind LyricsX particularly interesting. Here in this article, I’d share with you how LyricsX tracking the player progress in a unique and resource-saving way.

Common practices

Before we go to LyricsX, let’s first take a look at how the majority of other open source software work on this. In short, there are 2 major ways of doing it: rely on the player to send out update events, or query the player for progress at a regular time interval. Both ways are not ideal when it comes to the extreme cases where the user adjusts the player progress a lot, and the lyrics having a lot of lines in a short period of time.

The first way, relying on the player to send out progress update events, is not reliable because the update interval totally depends on the platform. Some platforms may issue updates several times a second, but some may just send once a couple of seconds. In the worst case, some lines might just be skipped over due to the long update interval. Even worse some might just lack of such feature exposed to the public, rendering this way unusable in those cases.

The second way, query the player for progress on a regular time interval, could indeed solve the problems mentioned above, but just in an ideal system where the query itself is as simple as reading a variable. In contrary, it is almost never the case in real life. Usually to query the progress, you had to go through some kind of RPC, or running a tiny script. Either way it is, it already sounds a lot complicated and resource-consuming, in a worse case, some may even need to send a request through the internet for the playing progress. Depends on the precision, you might want to run the query multiple times a second. But if a single query can already take half a second, it is almost impossible to achieve a better precision this way.

How did LyricsX do it?

The way LyricsX solved all the problems above is really cleaver. Instead of keep asking for the player for the current progress, it maintains a timer inside itself, and try to sync it with the player by using only a few basic events.

State

The program maintains a global state of the player, with the following properties:

Player status : Obviously, this can be either playing or paused. For the Stop state of some players, there is no track standby for playing, hence no timeline for lyrics, and thus out of our consideration.
When the state is playing:
- Internal t_0 time : This is calculated by current time in internal clock - player progress, serving as a reference point of the current play session. The internal clock can be anything that is fast enough to retrieve, like the system clock of the real world time, or a clock to measure code performance. We only care about the relative difference here.
- Playback speed : 1.0 for normal speed, 0.5 for half speed, 2.0 for twice the speed, so on and so forth.
When the state is paused:
- Playback progress : the progress of the player when it is being paused.

The global state is updated in the following player events:

Status change (on play, on pause, on stop)
Manual time change (on seek)
Rate change (on rate change)

You can see the event set we have is pretty basic, and mostly essential for such a software to run.

Sync the UI

With the player state in hand, it is much simpler to get the current player progress through just a simple system call and a calculation.

progress = ( t() - t_0 ) / rate

where t_0 is a function to get the current time in the internal clock.

Apart from that, since we now maintain a copy of the player state, we can simply assume that the player will continue the playback in the indicated speed as long as the state doesn’t change. With that, we can put more optimizations in place, such as generating animation sequence timelines ahead. All those preparation work only need to be done when the player state changes, which is much less frequent the update of playback progress.

In Lyricova Jukebox

In my attempt to port this logic in Lyricova Jukebox, I registered the play, pause, seeked and ratechange events, and for the internal clock, we used performance.now() for its high precision and the convenience to use in requestAnimationFrame().

For the timeline, LyricsX used a seemingly reliable internal method DispatcherQueue.schedule() provided by Apple. However, the scheduling function in web – window.setTimeout() – does not have the honor to have a high enough precision for us. We then resorted to have a requestAnimationFrame loop running when the media is playing. Thus, to start a loop when the player state turns to Start, and stop the loop when it turns to Stop.

In the requestAnimatioFrame loop, we check for the current time against the end time of the current keyframe. Since the loop is run every single frame in the browser, we keep the logic in the loop simple, and only advance a keyframe when the current frame ends. The frame pointer will only be updated in the hooks mentioned above to make sure the pointer is always pointing to the right frame.

If you are interested in the implementation details, you can take a look at the source code of LyricsX and its submodule MusicPlayer which is responsible for translating events from different players.

Alternatively, if you are more comfortable with React and TypeScript, I have ported this logic to my Lyricova Jukebox, which operates on the <audio /> tag of the browser.

Acknowledgement

Thanks a lot to ddddxxx, the main maintainer of LyricsX, for the brilliant idea and implementation of the software.

The post How LyricsX keeps track of progress of media players appeared first on 1A23 Blog.

Tonguess: Solver and Traffic Analysis

Eana Hufwe — Thu, 02 Jul 2020 09:03:33 +0000

Similar to the last post I did on a mobile game, this time again is another online battle game from Baton and QuizKnock — Tonguess (Android, iOS). In short, Tonguess is a word version of Bulls and Cows, once played by members of QuizKnock in a video.

How to play

Rules of Tonguess is simple: there are 2 players in a game, each player first comes out with a heterogram (word with no repeated letters) word in the dictionary of the game app. Length of the word to guess is decided prior to the game – either 3 letters or 4. Then they shall take turn to make guesses of the word of their opponent. Once a player makes a guess, the app will tell you how close you are to the answer. It will tell you 2 numbers:

Touch (a.k.a. cow, eat, B) is the number of letters you got correct but in the wrong place.
Get (a.k.a. bull, bite, A) is the number of letters you got correct and also in the right place.

Despite the word you choose at the beginning must be in the dictionary, you can use any combination of letters

Once a player guessed the word correctly in less rounds than their opponent, the game is won by the player. Similarly, it will be a draw round if both players got the word of each other in the same number of guesses.

Screenshots of the game, taken from the Google Play Store.

Solving the game

The dictionary

Dictionary plays a great part throughout the gameplay of Tonguess. The custom in-game keyboard used in both setting and guessing words provides strong visual hints on what words are in the dictionary. In the game settings menu in the home screen, there is even an option for you to go through all words in its dictionary and redirect you to a Google search on tap.

Screenshots showing how the keyboard changes when typing to show the available words in the dictionary

Hence, effectively every player can get hold of the dictionary and know whether a word is valid or not. What’s even better is that, after unzip the Android APK file, somewhere hidden in a binary file in the assets folder is the entire word list. That has later saved me a lot of time cracking the game.

Gameplay

As introduced previously, the strategy is pretty static. Apart from the answer to guess, nothing really changes from game to game, all other information like the word size and the dictionary are known prior to the games. It’s completely possible to build a single decision tree out of what we already have and use it across all future games.

While I was reading on relevant materials online on solving similar games, all of them were solving the number version of the game, which has the same range and domain, i.e. what you can guess is exactly what they can put as their answers. In this case, the path exploration process is more “symmetrical”, it needs to focus more on the number patterns in order to narrow down the candidates. Also, in this case, the exact number combination you start with doesn’t really matter, as wherever you start with a certain pattern, you can explore with the same strategy.

However, in Tonguess, the range and domain is not the same. The domain – what you can guess – is the entire n letter combination without replacement, 26 choose n; while the range – what you can put as an answer – is only limited to the number of entries in the dictionary.

Running through the binary file I found in the Tonguess APK with all dictionary words with a few Linux commands, I then got 2 files: one for those in 3 letters, and the other for 4 letters. They each had 711 words and 1799 words respectively, which is really not a lot comparing to the domain.

In this way, it probably makes more sense to use other strategies to build the decision tree statistically.

Building the decision tree

At the beginning, I was thinking of methods like MCTS, but due to the fact that back then my access to a decent computing power (i.e. my laptop) is greatly limited, I did not have the luxury to do trial-and-error with a Monte Carlo method that might not work after all.

Instead, I turned to other more brute force-y methods of building the tree. My first (and by far the only) idea is inspired by binary search. For every guess sent of n letters, there will be a maximum of 1+2+...+n+(n+1) possible feedbacks. Despite the nature of the game that more candidates are inclined to appear in those with less confirmed letters, with some even max out at 1, we want the distribution of candidates to be as even as possible for each and every guess. With my bare minimum knowledge from my Data Processing 101 course, I used standard deviation of branches as the heuristic function to decide which branch to explore.

The algorithm is rather straight forward:

Treat every game state as a tree node, with the current candidate words for answers. 

for each node
    go through every possible guess
        find the one with the best heuristic value
        record down the guess
        then build a node for each feedback to explore later
    repeat until a tree node has 0 or 1 candidate word.

One small detail to note: when there are multiple guesses share the same heuristic value, we want to choose the one that has a n-get-0-touch, as that would potentially save us one guess to win the game. The reason why we are looking at the actual candidate rather than checking if the guess is in the dictionary is that, sometimes a guess may be in the dictionary, but it could have been eliminated way up in the tree.

Coding it out

After I had the dictionary sorted out, I started write functions and scripts to help generate data I need along the way.

Before we get into the code, there are some terminologies I want to clarify here to better navigate you thorough the way.

candidates are the words that the opponent that could possibly put as their answer to the best of our knowledge at the current state;
a guess is a word we guess, or could guess in the game;
an outcome , or a feedback , is two numbers that we get after making a guess, could be formatted as x-get-y-touch, x – y, or (x, y);
the word length is the number of letters in the word we are guessing, i.e., either 3 or 4, usually denoted with n;
the dictionary size is the number of words in the dictionary of a specific word length, denoted with w_n.

The first building block is cal_outcome(guess, answer) -> Tuple[int, int]. As the name suggests, calculate the outcome from a guess and an answer. Apart from a few little quirks here and there, this is a pretty trivial function.

Coming after that, I built a lookup table for candidates grouped by outcomes for every guess at the initial state. This too was simple, but leaving all guesses and candidates in their string form has already taken me a few hundred MB just for 3 letter words in the plain Python Pickle syntax. I then tried to store the guesses as their indexes (in lexicographical order), and the grouping as a w-bit integer instead. (If a certain word is in that group, the corresponding bit is set to 1.)

With those little tricks, the mapping size has been reduced to 8 MB and 603 MB respectively. I have also used multiprocessing to speed up process.

Following that, I now can start building the decision tree. I almost completely followed the pseudo code laied out before, except using an external library to help counting number of 1s in a binary integer, and multiprocessing to parallelize the heuristic calcuation.

I couldn’t be bothered to think of ways to trim edges by looking into the nature of the gameplay strategies, my laptop has waaaay more time to compute than I have to put my code on it.

The time complexity of this one is about O(26 choose n} · n² log wₙ ). Sounds like a lot, but we’re not working on a competitive programming problem anyway. Looking at the CPU usage history graph, the 3-letter tree took a bit over 20 minutes, and the 4 letter one took about 4.5 hours.

Now we have the trees. Just one script to translate the numbers back to words, and another REPL-like script to move down the tree, it is finally ready to be played with.

What about entropy?

While reading on some more materials on this, I found out that entropy could be a better heuristic than standard deviation in this specifically setup with an inclined distribution.

I then duplicated my old script, replaced the heuristic function with entropy, and then a script to compare the two types of trees.

Surprisingly, with the help of numpy, the entropy trees were built almost twice as fast as the standard deviation one. The results of comparison shows that the 2 methods are basically on the same track, with the entropy one finishes a few dozen words earlier than the other.

Chart for 3-letter words
Chart for 4-letter words

There is an observable improvement on the overall number of guesses shifting towards the left by switching to entropy as the heuristic function.

The decision tree

A preview of the decision tree, click for the full version.

Taking a look at the tree we’ve got here, it does confirm with some of the strategies I find online to solve Tonguess.

Iterating through all possibilities n letters at a time when you have n-1 letters confirmed.^[1]

Part of the decision tree that guesses through all the *AST candidates.

Start a game with a word containing O, A or E.^[2] The generated tree started with the word OAT and TARE.

A web solver

Once we got the tree, which isn’t any big at all, we can use some JavaScript and a static web page to make a Tonguess solver.

A screenshot of the solver

The solver page and all source code mentioned above are available on GitHub.

blueset / tonguess-toolbox

Tonguess toolbox

Tonguess Toolkit

Files

build_mapping.py: Build guess-feedback mappings.
dict: Full dictionary text
dict3: Dictionary of 3 letter words
dict4: Dictionary of 4 letter words
entropy_tree.py: Build the entropy trees
entropy_tree.svg: Visualization of the entropy trees
entropy_tree3: Pickle of entropy tree of length 3
entropy_tree3.json: JSON of entropy tree of length 3
entropy_tree4: Pickle of entropy tree of length 4
entropy_tree4.json: JSON of entropy tree of length 4
guess.py: Word guessing helper functions
interactive.py: CLI solver
mapping3: Mapping for 3 letter words (need to build)
mapping4: Mapping for 4 letter words (need to build)
std_dev.py: Standard deviation helper functions
stdev_tree.py: Build the standard deviation trees
stdev_tree3: Pickle of standard deviation tree of length 3
stdev_tree3.json: JSON of standard deviation tree of length 3
stdev_tree4: Pickle of…

View on GitHub

Further improvements

In the future, this strategy of tree building can be further improved. Some of these potential improvements including:

Edge trimming strategies This could significantly improve the efficiency of the current tree building algorithm. For instance, in states where multiple letters are eliminated from the candidates, all guesses including these letters could potentially be reduced into a few due to the identical result they produce.
Put more weight on more common words The current strategy assumes all the words appear in the same frequency, which might not be the case for human. Incorporating the frequency may increase the chance of winning when playing against real players.
Explore more options There are potentially more info that we can exploit, or other strategies that could solve this game even faster.

Traffic analysis

As I explore further into the mechanism of the game, I realised that Tonguess – despite also being a game developed under the QuizKnock brand in the same genre – is designed with quite some difference from their first game Genkai Shiritori Mobile (GSM).

Tonguess keeps all the valid words locally, potentially due to the limited nature of dictionary, while GSM requires player to send the word up to their server to verify.
GSM has a global ranking for scores and words used and resets every month. Tonguess has no hint of any ranking at all, except a “rating” of your own which you can find in the app settings, and the rating of the opponent only available during the game.

Taking a chance, I fired up a packet inspector on my phone to capture the traffic.

As usual, those connected to Google servers, potentially for Play Services and IAPs, are HTTPS/TLS encrypted and not of our interest here. Yet the other connection, to connect for the actual gameplay, is not linked to Firebase anymore like GSM, but to a server claims to be gctok000.exitgames.com^[3], in an unusual port (TCP 4531). After some digging, it seems that this time they are using a game networking service called Photon Realtime. More importantly, the traffic is not encrypted, and that has enabled me to take a better look at what is going on behind the game.

The traffic

Photon Realtime uses a custom protocol on top of the classic TCP. Most of the traffic is ASCII data surrounded by binary components. I didn’t go too deep into decrypting those binary structure, but the ASCII payload itself is interesting enough for us.

When an opponent is first matched, the metadata of player is exchanged in a big JSON object, including:

Player ID (UUID)
Nickname
Rating as a float number with a long chunk of decimal places not shown to the player
Statistics
- Number of rounds won, lost and drew for 3-letter rounds
- And that for 4-letter rounds
Complete history of all unique 3-letter words the player has played (responsible to show the number of unique words played and the recent word history),
And that for 4-letter words.

A part of the packet that contains the big JSON object from the opponent.

It turns out, all these data are stored locally. For Android phones, they are URL encoded in one of the preference storage XMLs, and could be easily tampered with Root access. I guess, by storing the metadata of players locally, it could save a lot of server resources of the game maker, and thus reduce the operation cost. That could be why Tonguess comes with no global ranking compared to GSM.

After that, when the game actually starts, the game to guess was exchanged between players, again in plain text. This is actually very surprising to me. Despite that it could be a solution to avoid player missing the answer of their opponent once the connection is dropped, sending the answer in plain text enables players to cheat in the game in whatever way they like.

The packet that contains the word (ROCK) played by the opponent in a game.

They can even just straight out call out the answer at the beginning and win the game straightaway. But given there is no global ranking in the game, only a rating number – only shown when playing against others and can be changed even more easily – there could really lack of a motivation for anyone to do so.

Traffic after that isn’t particularly interesting anymore, just commands to update and switch the timer, delivering guesses to and fro, and heartbeat packets to ensure both players are still around (which could be what the communication library doing instead of the game devs).

Overall speaking, the game design and engineering is roughly up to the standard it aims to be — intelligently triggering, but not too complicated for both developers and players. Interest of the game may drop over time, but doesn’t make it too hard for it to maintain playable for returning players.

英単語ヌメロン「Tonguess」攻略・コツなど — またりんの限界日記
【世界最速！！！】Tonguess攻略② — しゅぎもん
000 can be any numbers from 009 to 012

The post Tonguess: Solver and Traffic Analysis appeared first on 1A23 Blog.

Why I am moving away from Disqus (totally)

Eana Hufwe — Tue, 21 Apr 2020 19:08:31 +0000

TL;DR : because their spam detection strategy is horrible.

Disqus is a comment service provider that gives free services to website owners to setup comment systems. I have started using Disqus a few years ago when I first setting up websites and blogs, because of its effortless way to add a comment box to seemingly any website, and the social log in feature. All seemed so good and peaceful until a day last year.

In an afternoon on the day, I got an email from Disqus suggesting my account is compromised and I need to reset my password.

Hi Eana,

Your Disqus account appears to have been used by an unauthorized 3rd party to post spam comments. We removed any newly posted spam comments and restored any previous posts that may have been edited.

To prevent further unauthorized access, we have disabled the password for your account. To set a new password, please follow the link below:

[link goes here]

We recommend that you change your password on other services that share the same password and use strong unique passwords for each. Having a shared password across multiple services is likely how the third party was able to access your account. A password manager like LastPass or 1Password can help prevent this type of access.

Regards,

The Disqus Team

The email from Disqus

At the moment when I saw the email, just a few minutes after it arrived, I logged in and changed my password. By the time there are already tons of spams sent with my account.

At that time things have already started to become weird. I tried to delete all the spam messages that the hacker sent, but all I got was the fake visual effect clicking on the delete button. Everything was back after refreshing the page. Back then I didn’t care much about that, since I thought I have recovered my account back, and they know that these spams were unauthorized, I should be safe.

But not soon after, when I started to comment on some of the websites using Disqus, I saw the problem. All comments, I mean all comments , no matter how harmless they are (or they may look like), all got the “detected as spam” label next to them. I had to manually contact the site admin through other methods to get an approval. I still didn’t care much about that at the time, as I don’t really comment on blogs that much anyway.

Then after a few months I started to feel like this should be a problem to be solved. I tried to look for channels to contact their support, but all I got is their community forum, which uncoincidentally happened to be yet another Disqus Channel. (Their customer support is only dedicated to enterprise users, which makes sense in someway. Surely nobody wants to waste manpower on users never gonna pay.) I tried to post my question there, and this time there are no marks about spam on the post (that mark usually appears next to the comment I send in blogs). I had my post there and hopefully looking forward to a reply from them.

Months later until recently I encountered another blog using Disqus, which reminded me about its spam disaster. I went back to my profile page, and saw that forum post itself is also “marked as spam”.

That forum post shown on my profile page.

Well, that’s really a bummer.

I then tried to comment on a few other posts trying to get the attention of the moderator, unsurprisingly all of them are barred for spam.

Two comments I made, manifestly legit comment, but barred as spam anyway.

By looking at the Spam tag in the forum, there are a lot of other users facing similar situations (obviously not the same, otherwise they can’t post there either).

There should be more, but I couldn’t find the search box of the forum.

With that, I have switched back to the built-in comment system as I move back from Hexo to WordPress, and it makes me to think twice when commenting on other sites running Disqus, as I had to contact the site admin through other methods to get my comment approved. Sometimes I might just tell the admin what I thought.

Not to mention other aspects that repel site owners from Disqus, like annoying UI/UX, ads mixed in comments, extra network request burden, reducing number of comments, bad moderation experience, user tracking, so on and so forth.

If you are using a static site, there are plenty of alternatives that you can try:

Utterances on GitHub Issues
Gitalk on GitHub Issues
Isso (self-hosted)
Schnack (self-hosted)
Commento (self-hosted, or pay-what-you-want cloud hosting)
Hyvor Talk (freemium cloud hosting)
CommentBox.io (freemium cloud hosting)
Valine (hosted on LeanCloud in China)
LiveRe (freemium cloud hosting in Korea)

Disclaimer: I’m just listing a few that I find interesting online. I haven’t used most of them, and I do not endorse any of them either.

Lastly, despite the fact that I’m leaving Disqus in most possible ways, I’m not deleting my account. Not that I want to give it a second chance (they didn’t give one either), but I hate to see the “comment is removed” label appear on top of my comment. I left them in hope that someone might find them useful, and I don’t want to disappoint people this way.

Tell me what you think about Disquz and its alternatives in the WordPress comment box below! If you have difficulty using this one, feel free to let me know via email, Twitter, Telegram, etc., and I’ll consider switching to yet another platform.

The post Why I am moving away from Disqus (totally) appeared first on 1A23 Blog.

DEV Community: Eana Hufwe

Rotated Background Patterns in CSS with SVG

Existing Approaches and Their Limitations

The SVG Pattern Technique

SVG Rotated Background Generator

Shift-JIS / UTF-8 文字化け解読：実はもうちょっと読めるかも

どれくらい解読できる？

文字コードの前提知識

実際に解読する

仮定その１：元データは有効な UTF-8 データ

仮定その２：元の文字列は日本語である

仮定その３：化け戻すとデータはロストしなければならない

仮定その４：難しい漢字は使わない（任意）

補足：１バイトと２バイト⸺デコーダ仕様の違い

Reverse engineering an IL2CPP NSO binary: Case study of Mojipittan Encore

Table of Contents

Forewords

Preparation

Extract contents

SciresM / hactool

hactool is a tool to view information about, decrypt, and extract common file formats for the Nintendo Switch, especially Nintendo Content Archives.

hactool

Usage

Unpack resource files

SeriousCache / UABE

Asset Bundle Extractor

Asset Bundle Extractor

Building

Preparing for decompilation

Perfare / Il2CppDumper

Unity il2cpp reverse engineer

Il2CppDumper

Features

Usage

Command-line

Outputs

DummyDll

ida.py

ida_with_struct.py

NationalSecurityAgency / ghidra

Ghidra is a software reverse engineering (SRE) framework

Ghidra Software Reverse Engineering Framework

Adubbz / Ghidra-Switch-Loader

Nintendo Switch loader for Ghidra

Ghidra Switch Loader

Building

Installation

Decompile the binary

Debugging with emulator

yuzu-emu / yuzu

Nintendo Switch emulator

yuzu

yuzu is the world's most popular, open-source, Nintendo Switch emulator — started by the creators of Citra It is written in C++ with portability in mind, and we actively maintain builds for Windows, Linux and Android

Compatibility

Development

blueset / MojipittanDictionary

Extracted dictionary data from Kotoba no Puzzle: Mojipittan Encore.

Mojipittan Dictionary

Files

Use WebVTT without actually using WebVTT: Another way to monitor playback progress of HTML Media Elements

Flexible and dynamic flow control of Azure DevOps YAML Pipelines using variables

Arbitrary gap between stages

Cut off a pipeline branch

Content-aware Infinite Scroll Loop using JavaScript

Repeating the list items

Scroll jumps to the opposite direction for a smooth loop

Scenario 1: No items are selected

Scenario 2: An item is selected

Smooth transition to the center when an item is selected

Updating the list with filters

Browser Compatibility

Extract and Visualize Your Telegram Group Network

Get prepared

Gather data

Construct graph

Visualize with Gephi

Adding metadata for Gephi

Some interesting analytics

Connected components and their sizes

Most connected nodes

yuzu is the world's most popular, open-source, Nintendo Switch emulator — started by the creators of Citra
It is written in C++ with portability in mind, and we actively maintain builds for Windows, Linux and Android