DEV Community: suntong

Google Docs no longer take free spaces in Google Drive

suntong — Sun, 06 Nov 2022 01:27:44 +0000

Everyone believes that their Google Docs sizes are not counted into the total 15G free spaces allocated, but it is no longer true!!

I found out today and it is quite a surprise to me and I have to triple-validate it before posting it here.

Google never said anything about it and it seems that most people aren't unaware of such new situation. Take this article for example, which is only published this year at MAY 25, 2022, still believe otherwise:

These Things Do Not Count

Google does not count any word documents you create in Google Docs, any spreadsheets from Google Sheets, any presentation slideshows from Google Slides (thankfully), any forms or surveys from Google Forms, or any information for Google Sites.

Moreover, as of Saturday, November 05, 2022, this is what wikipedia says about Google Drive:

Google gives every user 15 GB (1 GB = 1 billion bytes) of free storage through Google One. Google Docs, Google Sheets and Google Slides files do not count towards the storage limit.^[35]

YET, they do now!

Be aware!

st - the suckless simple terminal, v0.9

suntong — Sun, 16 Oct 2022 19:43:38 +0000

TL;DR

This Debian package does provide a scrollback buffer, it also supports color emoji, and it shows Chinese beautifully:

Why bother

For decades, I settled with urxvt to display Chinese, despite all its quirks, like surprising hot-keys and not supporting color emoji.

However, it is its "helpful" anti-aliasing font effect under WSL that become the last straw that broke the camel's back:

As I was not able to get rid of the above anti-aliasing effect under WSL.

Why not Alacritty

I agree with the comment from reddit that:

It's Ok to use Alacritty,

if you really want something a bit more bloated, and don't mind using rust software.

Alacritty
Like GNOME, it may look nice, but it consumes way too much RAM for a barebone terminal emulator IMO.

$ ps -eo comm,rss,pcpu | grep -E 'xterm|alacritty'

xterm 13052 0.0
alacritty 38884 0.1

Depending on your distribution, the packaged alacritty may be fatter than others. Here on Fedora (rawhide) is the new GNOME 42 terminal...

ps -eo comm,rss,pcpu | grep -E 'terminal|kitty|st'
kitty           83668  0.5
alacritty       69668  1.3
gnome-terminal- 61260  6.4
st              10172  0.3 (unpatched)

I don't mind it is 4+ the size of st as long as I can get a precompiled Debian/Ubuntu package (as I really don't want to compile any packages myself). However, I gave up alacritty for the exact reason posted under here:

Something to note when using a gpu based terminal. Having the terminal emulator open will make the gpu ramp up a few power states. My 1080ti goes into power state 3, which results in an extra ~30 watts at the wall over using st, urxvt, etc.
Don't get me wrong, alacritty is amazing, but it's not worth the extra power draw for me.

I.e., despite how amazing it is, I'm not willing to let my laptop run hot all the time and reduce its battery time just for some fancy visual effects.

Why not sticking to the official precompiled Debian/Ubuntu package?

Despite how I dislike compiling packages myself, I cannot bear with the official precompiled Debian/Ubuntu package because it does not provide a scrollback buffer, and it crashes when displaying color emojis.

My precompiled Ubuntu package

Because I really don't want to compile any packages myself, and because the only way to get st is to compile it yourself, I'm providing this precompiled Ubuntu package in hoping that it'll help someone like me. It is built right from source git repo, thus it now supports color emoji, and it'll always give you the latest version. Besides, I added the patch so it does provide a scrollback buffer.

Here is how to get it installed under Debian 11 bullseye (Ubuntu would be similar but simpler):

Check out ppa:suntong001/ppa at https://launchpad.net/~suntong001/+archive/ubuntu/ppa:

This PPA can be added to your Debian 11 system manually by copying the lines below and adding them to your system's software sources.

deb https://ppa.launchpadcontent.net/suntong001/ppa/ubuntu focal main 
deb-src https://ppa.launchpadcontent.net/suntong001/ppa/ubuntu focal main

Then, after apt update, do:

apt-cache policy stterm libxft2
apt install stterm libxft2

Recheck:

$ apt-cache policy stterm libxft2
stterm:
  Installed: 0.9.1~20221004~ubuntu20.04.1
  Candidate: 0.9.1~20221004~ubuntu20.04.1
  Version table:
 *** 0.9.1~20221004~ubuntu20.04.1 500
        500 http://ppa.launchpad.net/suntong001/ppa/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
     0.8.5-1 500
        500 http://deb.debian.org/debian bookworm/main amd64 Packages
     0.8.4-1 500
        500 http://deb.debian.org/debian bullseye/main amd64 Packages
     0.8.2-1 500
        500 http://deb.debian.org/debian buster/main amd64 Packages

libxft2:
  Installed: 2.3.6-1
  Candidate: 2.3.6-1
  Version table:
 *** 2.3.6-1 500
        500 http://deb.debian.org/debian bookworm/main amd64 Packages
        100 /var/lib/dpkg/status
     2.3.6-1~202209251622~ubuntu20.04.1 500
        500 http://ppa.launchpad.net/suntong001/ppa/ubuntu focal/main amd64 Packages
     2.3.2-2 500
        500 http://deb.debian.org/debian bullseye/main amd64 Packages
        500 http://deb.debian.org/debian buster/main amd64 Packages

That's it. Enjoy 😃.

Transforming json data with easygen

suntong — Fri, 31 Dec 2021 04:00:35 +0000

Filtering

Take a look at the vast amount of data a github API may return, e.g.: https://api.github.com/search/repositories?q=go.

If there is only few key data that we want, then the first thing to do is to filter out the unwanted. Can easygen do that? Yes, of course. Given the power of Go template engine, everything is possible. However, such filtering is imperative programming if done in easygen, especially when doing vertically filtering (skipping unwanted records). So it is better to use the declarative approach, using tools like jp or jq instead. Since that is outside of the scope of this article, we'll skip it and move on to the next section.

Presenting

Take a look at the end result first, at https://github.com/go-sqlparser/current/wiki/List:-Currently-Collected-Projects:

Here is what happening behind the scene -- the mark down source code that renders above:

Note that

the project name keyword has been repeated five times in the mark down source code.
there is a TOC index that links directly to the detailed content. I.e., each project is repeated twice, first listed briefly in the TOC/index section, then full details in the following section.
the PrimaryLanguage is therefore need to be repeated in two sections as well.

Such repetitive tasks are best leave to tools to automate it, and easygen is such code/text auto generating tools. The mark down code generation template is as simple as this:

$ cat repos.tmpl 
## Currently Collected Projects

### List
{{range .Data.RepositoryOwner.Repositories.Edges}}
- [{{.Node.Name}} ({{.Node.PrimaryLanguage.PrimaryLanguage}})](#{{.Node.Name}}){{end}}

### Details
{{range .Data.RepositoryOwner.Repositories.Edges}}
<a name="{{.Node.Name}}"/>**{{.Node.Name}}** ({{.Node.PrimaryLanguage.PrimaryLanguage}})  
{{.Node.Url}}  
{{.Node.Description}}
{{end}}

To put them together

So in order to come up with the above end result, I need to

download json data via API
filter the json data with the jp
then present the json data in human friendly mark down format with the new easygen that can read from stdin.
in the human friendly form, if I need to convert the long number of project size from bytes to the size in KB or MB, and I can make use of easygen's built in calculation in Go template supports.
there are lots of other transformation support functions already builtin inside easygen, check out the full list with sample usage and results here.

UPDATE:

project size added to list details:

and here is the updated code generation template:

$ cat repos.tmpl 
## Currently Collected Projects

### List
{{range .Data.RepositoryOwner.Repositories.Edges}}
- [{{.Node.Name}} ({{.Node.PrimaryLanguage.PrimaryLanguage}})](#{{.Node.Name}}){{end}}

### Details
{{range .Data.RepositoryOwner.Repositories.Edges}}{{$mb := sprintf "%.2f" (divide .Node.Size 1024)}}
<a name="{{.Node.Name}}"/>**{{.Node.Name}}** ({{.Node.PrimaryLanguage.PrimaryLanguage}}, {{.Node.Size}}KB/{{$mb}}MB)  
{{.Node.Url}}  
{{ coalesce .Node.Description "No description" }}
{{end}}

Credit

The cover image is obtained from imd.org.

What is the "XSLT" equivalent for JSON? Here is the new answer

suntong — Thu, 30 Dec 2021 04:33:55 +0000

Prefix

What is the "XSLT" equivalent for JSON?

This is the question that people had been asking more than 10 years ago. By then the xslt is still the number one answer.

It's high time to take a look at the new kids in town now.

What is "XSLT"?

XSLT stands for XSL Transformation. It is used to transform XML documents into into various other types of document (like transforming XML into HTML).

It's most important purpose is transform data, from XML form into a format more presentable to human beings.

XSL is to XML is similar as CSS is to HTML. XSLT can be viewed as the rendering engine.

What are the new choices now?

The second choice in the above answer had been jq, and I agree that it is very powerful.

jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text. There are install packages for different OS.

But comparing to jsonpath it is more difficult to learn to most people, which brings us to the first tool to recommend this time -- jp, because

jp is as simple as jsonpath but as powerful as jq.

How about the rendering engine part?

The problem of both jq or jp is that although they can do data projection in any direction (doing data filtering in any way), their purpose are still focusing on data transformation, but when talking about rendering json data into a format more presentable to human beings, then there is no better tools than easygen. In fact, the easygen is a universal code/text generator that not only works on json data but it can take in data defined in YAML format as well. It can be used as any text (or html, or any other forms) generator for arbitrary purposes with arbitrary data and templates. It is built on top of the powerful Go template engine.

To put them together

So the other day, I

download json data via API
filter the json data with the jp that supports filtering by regular expression (hint: do you know any json filter that when given a video's file name, can tell a tv-series apart from a movie?)
then present the json data in human friendly format with the new easygen that can read from stdin.
in the human friendly form, I need to convert the long number of size in bytes to the size in MB and GB, and I had to search for how to do calculation in Go template, but then was delighted to realized that easygen already supports the calculation in Go template long time ago.

I'll give a more concrete example in the next article.

To me now,

easygen is to json what xslt is to xml, but much more powerful and versatile.

Credit

The cover image is obtained from imd.org.

Maintenance free system multi booting

suntong — Fri, 29 Oct 2021 03:18:03 +0000

The journey continues

First, recap from the previous article, a single grub boot menu entry can be as simple as:

With it, we don't need to hunt for the UUIDs, and we don't need to remember to mount boot partition and update the boot entries any more. It can boot to whatever latest kernel it is, and it will still work even we reformat the partition and install a new Linux, and it will still work even we change the distro from Debian, to Mint, Arch, Fedora, or openSUSE etc.

In this article, we'll first look at those few more boilerplates that enable you to only write the above few lines and everything will work out magically. Then we'll move on to how to make things even simpler, with the help of easygen.

The supporting files

To break free from the rigid format requirement of 40_custom and allow people only write the above few lines, or to order boot entries in the order we want, instead of being determined by their partition orders, we need to build our grub.cfg file from ground up ourselves. And, it is not that scary either and could be really straightforward. Here are all the code it needs:

The grub.cfg file that replace grubs default .cfg file. Deliberately made so simple so that it can be easily restored if it was accidentally overwritten.

The grub-main.cfg that the grub.cfg file actually calls. The idea comes from grml. We can see that it is sourcing a header, a footer, and the main/real .cfg files, like the entry shown above.

Arrange those .cfg files so that their order show up in the order you want in the grub menu. For example, this is how I arrange my grub menu:

I.e., in my preference, I want my grub menu to be in the order of:

My current Linux
Live Linux from ISO, in case I need to check/repair anything
My Windows
My old Linux versions

They are ordered from most probably used to the least probably used. Changing the order is simple, just shift them around in the .cfg text file.

The grub menu's look-and-feel is defined in the header, and the "Shutdown / Reboot" is defined in footer, showing below respectively.

ISO Booting

The bonus and prestige of the whole article, here is how ISO files are automatically discovered and listed:

It can automatically discover and list whatever ISO available on disk:

and boot from them:

The red background color is chosen for the grub menu, to symbolize the live system, and also as the notification sign that this will be the last grub menu level, after which the system will be in booting mode.

Can it be even simpler

So far, we can see that using some simple scripts/configs, we can let the grub to show whatever we want it to show, in whatever order we want as well. Now,

Can it be even simpler?

As pointed out in the "Conclusion" section in the last article, the only things that matter in a grub menu entry are the 'Debian GNU/Linux' and the 'hd0,gpt2', the rest are just the boilerplate that don't normally need to be changed.

What if I only want to specify the 'Debian GNU/Linux' and the 'hd0,gpt2' part, and let the machine to generate the grub menu entry/entries for me? -- Not a problem, if with the help of easygen.

This is all what you need to specify:

Feed that to easygen as:

easygen os2grub-1

then you'll get:

The secret is the easygen's template:

Customize it to whatever you like.

But, most people not just has a single boot menu. How do you deal with that? Simple -- with another small tool of grub2yaml.sh:

Just run it with

grub2yaml.sh /boot/grub/grub.cfg | tee /tmp/myOSs.cfg

Slightly edit it, and you'll get more entries like this:

With multiple entries like that, easygen really starts to shine. You can get the following without a drop of sweat:

Enjoy

IMHO, the way grub2 generates boot entries automatically, at least as of this article is written, is still a big mess. You may not agree with that, but I do enjoy my grub menu system to be plain and simple, straightforward and out of my way. And, I enjoy my freedom to hand-craft my own boot menus again, just like the good old days, but without an extra spare letter to waste.

Hope you enjoy that as well.

Cheers

Journey towards a maintenance free system multi booting

suntong — Thu, 28 Oct 2021 00:05:35 +0000

Problem statement and goal

This article tries to simply the situation for people who dual boot/multi boot their computer systems, be them Debian, Mint, Arch, Fedora, openSUSE, etc, or Windows.

For multi booting all the operation systems, I have a standalone boot partition as well as an UEFI ESP partition. From there, any operation systems on my disk can be booted. A Debian boot entry looks like this:

Let me stress again that this is just one single entry. There are also several "Advanced options" that goes with each Linux system, so my multi-boot grub.cfg file looks scary, because it is humongous. But that's not the only problem.

Problem #1: Because it is in a standalone boot partition, I normally don't need to mount it, but whenever I got my kernel manually or automatically upgraded, I have to remember to mount this partition and update the boot entries there. Otherwise,

I will not be able to boot up my system again, if the old kernels are somehow/automatically purged.
I will not be able to take advantage of the new kernel, even if they have been installed.

Problem #2: I like to bring my 40_custom up to the top of my menu selection, instead of down at the bottom.

Problem #3: I like to order my boot entries in the order I want, instead of being determined by their partition orders.

Although grub can generate boot entries automatically, all above problem are blind spots of grub, and you have to come up with your own ways of dealing with them.

The good news is that the grub.cfg file does not have to be so humongous, scary, and messy. At the end of this article, you'll find that things can be pretty simple. Scroll to the bottom to see if you don't want to go through the discovery journey that leads towards it. I.e. you'll find that

Things can be so simple that people can start writing/maintaining the grub boot menu by hands now. And in the next blog, we can see that things can be even more simpler, with the help of easygen.

Let the journey begin -- we'll going to go through a series of steps to simplify the grub.cfg file so that in the end, they can be edited manually. Now, step number one,

Remove the "Advanced options"

Let's take a look how big my grub.cfg file is:

$ wc /boot/grub/grub.cfg
  355  1338 15006 /boot/grub/grub.cfg

Over 350 lines! Let's first simplify the grub.cfg file by remove the "Advanced options" menu entries, as I've been using Linux for over 20 years, and not a single chance had I ever need to dip into those options. Moreover, the grub menu can be easily edited if I want, say to boot into single mode. They can go for sure:

$ sed '/^submenu .Advanced options for /,/^}/d' /boot/grub/grub.cfg | wc
    183     581    5602

The file size almost dropped in half and now is under 200 lines.

Remove the unnecessary if conditions

Look at the following lines in the above posted grub boot menu:

Has anyone ever wondering what it is doing?

Well, it turns out that

the conditional code checking for this feature is pointless in all modern Linux distributions

So why still keep it around?

These if conditions are second that need to go. After that, a single boot entry will look like:

Remove the unnecessary module loading

Now, why are there so many insmod statements in each boot entry? My computer contains either Linux or Windows, and they reside all in GPT, as ext4 partitions. I.e.,

All my Linux will need the same modules, so it doesn't make sense to repeat every one of them in each individual boot entry.

Can I load all the modules I need only once at the beginning, and save each individual boot entries from repeating themselves?

It turns out that I can. So now my Linux boot entry looks like this:

A little help please

It'll be tedious to go through each boot entry and do the above clean up, so a little help from a script will definitely be the way to go, and here it is:

Toward maintenance free booting

Now it is time to tackle Problem #1 -- whenever my kernel is upgraded, I don't want to

mount my boot partition, and
update the boot entries there

I want my boot partition's boot entries to be able to survive such change and always boot to the latest kernel, whatever they are. Is that possible? Well,

It sure can, using the trick like this:

Maintenance free, at last!

Look at the linux /vmlinuz line, there is still a root=UUID=xxx parameter there. Now what if I reformat the partition and install a new Linux? The UUID will surely change and the above boot entry will surely not be able to survive such change.

This hurdle is the most difficult of all. Luckily someone has already looked into such problem, but sadly, nobody else even care, judging from the up-votes from the SO question and answer. Anyway, here, the prestige of the whole article, here is the solution:

The probe -u $root --set=rootuuid line will set the $rootuuid variable, then it get passed to the next linux /vmlinuz line.

Hooray~~, problem solved!!

One step further

Now, take a closer look at the search --no-floppy --fs-uuid ... line.

What does it do?

It actually set the $root grub variable according to a fixed UUID value on that line, and therefore change the boot partition, which will undermine the whole effort of our above attempt.

That will surely need to go away.

Summary of the journey

The above discovery journey was a step by step one and it has been logged here:

The boot entries were build from the ground up and move step by step to the top. All entries are verified to be working fine.

Conclusion

Once upon a time, people write their own boot menu. But grub2 came alone and people just gave up such practice and rely solely on the machine.

The journey in this article show that things do not have to be that way as the only way -- we can still write our own boot menu, just do not let the humongous file size scare you off, as a single boot menu entry now can be as simple as:

Most of which are boilerplate that don't need to be changed. I.e., the only things that matter are the 'Debian GNU/Linux' and the 'hd0,gpt2'. We don't need to hunt for the UUIDs, and we don't need to remember to mount boot partition and update the boot entries any more. It can boot to whatever latest kernel it is, and it will still work even we reformat the partition and install a new Linux, and it will still work even we change the distro from Debian, to Mint, Arch, Fedora, or openSUSE etc.

There are a few more boilerplates that need to be in place so that you can only write the above few lines and everything will work out magically, which will be covered in next article, with the help of easygen.

Settling the editors key bindings war

suntong — Sun, 20 Jun 2021 04:34:59 +0000

This SO question, Key bindings war in my brain, best summarized the problem I'm facing and am trying to solve:

How many key bindings are there in your brain? I personally use Intellij Idea as my primary IDE for development, but sometimes switch to Eclipse and Netbeans a bit, and sometimes I use Jedit, Notepad++, Emacs, Vim.

I am always trying to find a perfect key bindings that I can use crossing all the editors. What's your idea to solve this problem?

The question was asked over 10 years ago, and there is still no perfect solution to the problem. What's worse is that there is now another big player in the field now -- we need to add Microsoft Code to the above long list as well now.

There may be no perfect solution for anyone, but for me, having to constantly switch between Emacs, Jed, Goland, IntelliJ Idea, and Microsoft Code, I'm going to let them all to use the same keyboard mapping and bindings.

The most difficult part is not Goland, IntelliJ Idea, and Microsoft Code, contrary to people might think, but Emacs itself, as Emacs itself has no official standard for key mapping and bindings. For example, the other day I just realized that I have three different way for comment-or-uncomment: C-c ;, M-#, and M-;. Same for indents lines and/or region, I have defined at least three different ways as well.

Both Microsoft Code and IntelliJ has Emacs compatible mode which use Emacs key bindings. This post just collect them into a single place and put them side by side (if dev.to is restricting the blog width so you cannot see the full table width, you can also view it here):

PS.

If you are interested in how these key bindings are collected, check out

Collecting Awesome-Emacs Keymaps, and
Collecting IntelliJ-Emacs Keymaps which covers combining the two as well

How to Install Google Services Bundle Yourself

suntong — Mon, 26 Oct 2020 02:52:08 +0000

When I got my new Xiaomi phone, I wasn't prepared for the fact that installing Google Play and Google services would be this hard, because almost every youtube video, every blog, every technical article shows you that it is really simple to do so. All of them shows similar steps like this:

Launch the Mi App Store
Search for Google
Scroll down to the bottom of the list and select this option:

Install the Google Installer app from its icon, and follows the prompts for the rest of the steps.

This is the first thing that I did, however Google Installer app complains that my phone is not rooted yet, so it will not go any further.

Unlike other blogs, I don't thing the following are possible ways out:

Return the phone to sender and purchase a Global model
Unlock the bootloader and install a Global ROM

So the only option I have is to Install Google services myself. It looks really intimidating, but having done that, I realized that it is not a big deal at all, and will not be afraid to buy Chinese phone that don't have Google services pre-installed any more.

The phone that I have is a Redmi 10X 4g running MIUI 12. The steps will be similar for all Xiaomi/Redmi phones, and now I believe even for Huawei phones as well.

The most important trick is

to installed with the correct minimum set of required apps,
in the correct order
and from the correct package with correct version

If any of above is not right, then you might not have a good installation in the end. So here is the log of my attempt that works in the end.

In short, install the following required services in this order:

All are available from

https://www.apkmirror.com/apk/google-inc/

The only problem is that when you get to the above download pages, most people would be intimidated by the selection of device’s DPI, architecture, screen size, and other specs. Luckily, apkmirror has that covered as well:

How can I find out my device’s DPI, architecture, screen size, and other specs?

After trying numerous system info apps, we are currently recommending Droid Hardware Info. It contains the DPI (Device tab), architecture (System tab), Android OS version (Device tab) as well as a plethora of other info.

So get that installed first (com.dama.hardwareinfo.apk), and know your device. The rest will be easier.

The next problem is to choose & get the correct package with correct version. Here is mine. I believe it will work for all new Xiaomi/Redmi phones, and even new arm based Huawei phones as well. Here is the list, the package name, download url, and the actual .apk that I downloaded (which shows the architecture, dpi, and version).

Google Services Framework, https://www.apkmirror.com/apk/google-inc/google-services-framework/, com.google.android.gsf_10-6475670-29_minAPI29(nodpi)_apkmirror.com.apk
Google Account Manager, https://www.apkmirror.com/apk/google-inc/google-account-manager/, com.google.android.gsf.login_7.1.2-25_minAPI23(nodpi)_apkmirror.com.apk
Google Play Services, https://www.apkmirror.com/apk/google-inc/google-play-services/, com.google.android.gms_20.42.14_(120400-338133832)-204214037_minAPI29(arm64-v8a,armeabi-v7a)(nodpi)_apkmirror.com.apk
Google Calendar Sync (must), https://www.apkmirror.com/apk/google-inc/google-calendar-sync/, com.google.android.syncadapters.calendar_6.0.44-267540251-release-2016267990_minAPI21(nodpi)_apkmirror.com.apk
Google Contacts Sync (must), https://www.apkmirror.com/apk/google-inc/google-contacts-sync/, com.google.android.syncadapters.contacts_10-5771379-29_minAPI29(nodpi)_apkmirror.com.apk
and finally the Google Play Store, https://www.apkmirror.com/apk/google-inc/google-play-store/, com.android.vending_22.4.28-21_0_PR_338099147-82242810_minAPI21(armeabi,armeabi-v7a,mips,mips64,x86,x86_64)(nodpi)_apkmirror.com.apk

If you get everything right, then you will sure to have a good installation in the end.

One thing though, I was having a bit trouble installing Google Maps.

The current default is called Google Maps to Go, and it runs only in the browser. It was super slow, even on my brand new phone of the latest module, and I read review people are also saying it is slow, and recommends the classic Google Maps instead.
Thus I went with the classic Google Maps instead, however, it crashes by itself soon after it is launched. I read review and it is happening to other people as well.
So I have to rely on apkmirror.com again for an older version of Google Maps.
The first old version (2nd in total) that I tried, is version 10.50.4, but it crashes as well.
Only when I went back to version 10.49.3, did it start without crashing.
I.e., overall, the .apk that I kept is, com.google.android.apps.maps_10.49.3-1049301244_minAPI21(arm64-v8a)(400,560,480,640dpi)_apkmirror.com.apk

That's it.

Good luck!

Are you still coding go cli handling by hand?

suntong — Thu, 15 Oct 2020 14:10:25 +0000

Are you still coding your go cli handling by hand? Time to stop and rethink better ways to do it now.

Take a look at this go command line handling file:

The help text "path to serve files from" shows up three times:

$ grep -1 'path to serve' OpenSesame_cliDef.go
        Port string // listening port
        Path string // path to serve files from
        Help bool   // show usage help
--
        flag.StringVar(&Opts.Path, "path", "./",
                "path to serve files from")
        flag.BoolVar(&Opts.Help, "help", false,
--

const usageSummary = "  -port\tlistening port (OPENSESAME_PORT)\n  -path\tpath to serve files from (OPENSESAME_PATH)\n  -help\tshow usage help (OPENSESAME_HELP)\n\nDetails:\n\n"

If coding by hand, most probably the first occurrence will be omitted. But that's kind of pity, as the help text is helpful in explaining what exactly the Path string variable is for:

        Path string // path to serve files from

Also, if coding by hand, the second and third occurrences will very easily get out of sync if not be taken care of all the time.

All these violate the DRY (Don't Repeat Yourself) principle. How to avoid it? The answer is this file:

from which we can see that the help text "path to serve files from" is defined only once but get automatically put into three different proper places.

This is the go cli handling code auto generation we are talking about. Look at the above two files, which one is more easier to maintain? With cli handling code auto generation feature out there, don't code it manually next time when you do cli handling again.

What if you already have a manually written cli handling code? Well, converting it to automatically generated is not that hard at all. Here are just two simple steps,

introduce config.go, which ports the existing cli handling from existing code into auto-generated config.go file.
implemented -d, -c, & -help, which adds more business logic handling for the newly added command line flags/parameters.

Very simple, no sweat at all. All thanks to the auto-generated config.go file.

OK, now comes to the question, how is the config.go file auto-generated? Let me answer it with another example, that starts everything from the beginning, with all the following explanation borrowed from here

This section will work through every steps how the program is created.

Initialize an empty project. The _proj.yaml file will later be used to
Initialize project wireframe using the standard Go flag package, by
1. Edit the _cli.yaml cli arguments definition file
2. Then do go generate (enabled like this), or directly invoke the _cliGen.sh script
3. Finally do go build
Provide the initial OpenSesame functionality
Update OpenSesame code to make use of the auto-generated cli wireframe
The auto-generated cli wireframe code are not set in stones, they can later be further amended, just like the real-world cases. For e.g.
- Update cli definition of port value
- Update usage doc
- Enable UsageSummary
- etc.

This finished a full Initialize -> Make use -> More changes to cli definitions cycle.

Make sure to check out the rest of the above quoted wiki, Go command line flag handling code auto generation, from which you can see that using the same .yaml driven file, not only the go standard flag command line handling code can be automatically generated, but viper and cobra as well, and all the way to the full-featured cli code that can set the argument values at three different levels:

So the priority of setting the Host value is, from lowest priority to the highest:

self-config file
environment variable
command line

Three different levels.

Sprint start your wechat bot with wechaty padplus

suntong — Thu, 26 Mar 2020 04:03:42 +0000

Almost all online chatting apps provide APIs to access their functionalities, but wechat is an exception. There are no official functionalities/supports on either APIs or bots. But that doesn't deter people from trying to automate those repetitive tasks in wechat with bots, all thanks to those people who had jumped through the hoops backward to provide programmable access to wechat functionalities. One of the best of such platform is padplus under the wechaty framework (believe me, I know it the hard way).

This article will illustrate how easy to get started with wechat bot programming using wechaty padplus. The existing documentation is already very helpful, please check out the following two if you haven't done so:

Both are very good, so what covered there will not be repeated here. Instead, we'll dive directly into practical aspect to get you hit the road and run.

The padplus readme gives a generic typescript code as a guideline on how to start your script. You cannot directly run it as there are still customization you need to make before stating it( e.g., replacing the padplus token with yours). To make it even more simpler, here is a code that you can grab and run right away:

What it does is to get the padplus token from the environment variable WECHATY_PADPLUS_TOKEN, so all you need to do is to set it before running the script. To set and run in one step, here is how to do it under linux:

WECHATY_PADPLUS_TOKEN=puppet_padplus_xxxxx ts-node starter-padplus-bot.ts

Me personally don't like TypeScript very much. I know it is good for corporation that help preventing people from shooting themselves in the foot, but I still prefer the plain old JavaScript, the ES6 actually, as I believe with proper tools (like eslint, 'use strict', etc) ES6 can be a good language under the care of a good programmer. The extra bell-and-whistles, and thus the extra overhead of TypeScript do not appeal to me much, as I write only for myself.

If you prefer JavaScript as well, I've prepared the JavaScript version of the starter-padplus-bot file (based on ding-dong-bot.js):

As you can see it is very close to a full-fledged bot script. It

has the shebang so that it can be started as a command under Linux.
has all the essential event handler examples
has logging code builtin and ready to be used
has logged all incoming message on console so that you can start dealing with them right away
has interactive behavior builtin so that people can check from the wechat end to verify if the bot is alive or not

Here is how to start it (under xterm, somehow I cannot get the small version of qrcode working, but only the normal sized, as shown in the picture. I think it might be a bug BTW):

Once you can automate things for wechat, the application scenarios are endless. So start automate your wechat and make it alive!

And last but not least, the highlight of the whole article, if you are willing to contribute your code back to the wechaty community as open source project, you should be able to get a padplus token for free as long as you continue to contribute using many different ways including just blogging it.

Your favorite online chart and embedding them to dev.to

suntong — Sat, 04 Jan 2020 14:37:00 +0000

First of all, here are some links to embedding charts online:

However, after carefully examining the situation, I found that we cannot do any of above here at dev.to -- my conclusion is that people are not able to embed online charts to dev.to yet, not even HTML iframes, see:

HTML Embeds for Dev.to posts #1445

bnb posted on Jan 02, 2019

Describe the solution you'd like: I'd like to see HTML embeds in addition to social sharing, along the lines of how you can embed Tweets or YouTube videos with simple HTML (plus a script or iframe, respectively).

Describe alternatives you've considered:

Adding oembed, Embedly, etc.
- Isn't viable for every site, especially not ones that I contribute to but don't own.
Hand coding a pretty embed
- Not enough of a reason on the user end to do this.

Additional context: Would love to include a pretty dev.to embed in a blog post that is being posted in a couple days but unfortunately it doesn't currently exist.

View on GitHub

However, charts are the most powerful way to convey ideas -- they say a picture is worth a thousand words.

Hope we can have online charts in our posts soon.

hiccup -- Edit will always bring me back to an older version

suntong — Sun, 21 Jul 2019 20:44:18 +0000

Please take a look at
https://dev.to/suntong/grub4dos-to-boot-iso-disk-image-57ho
in which there are three reference entries at the bottom.

However, if I do "Edit" from that page,

the reference will become my first attempt:

Ref: https://ubuntuforums.org/showthread.php?t=2398574

Tried many times, including closing it and opening it in a new browser window, but the results are always the same.

What is so special about my this post, or there is a system wise hiccup in dev.to? Thanks