DEV Community: Hollow Man

My Igalia Coding Experience 2023 I & II at Wolvic

Hollow Man — Sat, 17 Feb 2024 17:46:10 +0000

Wolvic is a fast and secure browser for standalone virtual-reality and augmented-reality headsets. ex. Mozilla Firefox Reality.

Project summaries

Develop VR Browser, refactor Android deprecated methods
Address user issues. Implement UI, graphics, browser, and openXR related features
Contribute to the majority of the features available from v1.4.2 - 1.6:

List of things I have done

PRs opened/handled

(merged) #773 Fix build with JDK 17
(closed) #787 Fix and Check warnings / deprecation notes in the current build
(closed) #796 Fix deprecated android.inputmethodservice.Keyboard and keyboardView
(merged) #811 Fix and Check warnings / deprecation notes in the current build
(merged) #812 Fix deprecated android.inputmethodservice.Keyboard and keyboardView
(merged) #814 Fix dependabot.yml syntax
(merged) #820 Upgrade Android dependencies
(reviewed) #829 Initial flow for mainland China
(reviewed) #831 Use androidx's PreferenceManager
(reviewed) #621 Add some dependency conflict resoluton strategies
(merged) #838 Fix keyboard icon displaying
(reviewed) #824 [l10n] Update translations to Chinese (Simplified)
(merged) #843 Support per-architecture dependency substitutions for Gecko
(merged) #844 Implement POST resubmission confirmation
(closed) #848 Homepage bypass cache so that it can work corectly after a language change
(reviewed) #816 Bump com.android.tools.build:gradle from 4.2.2 to 8.0.2
(reviewed) #781 Remove WaveVR build dependency
(merged) #852 Fix several issues related to Download List
(merged) #853 Use left alignment for the download Confirmation Dialog
(merged) #854 Modernize deprecated setSystemUiVisibility(int) and related flags
(merged) #855 Make keyboard follow the system locale if never manual select
(reviewed) #856 [HVR] Make PlatformActivity inherit form Activity again
(reviewed) #819 Bump net.lingala.zip4j:zip4j from 1.3.2 to 2.11.5
(closed) #859 Ask OpenXR runtime for available GL formats options
(closed) #860 Replace kAverageHeight with XR_REFERENCE_SPACE_TYPE_STAGE
(merged) #862 Fix Android tests
(reviewed) #866 Remove overlay extension support for Pico
(reviewed) #640 Ana2k/date time picker
(merged) #877 Implement DateTime picker Dialog
(merged) #878 Fix to keep the selected options for <select>
(reviewed) #876 Meta store fixes
(merged) #879 File:// uri navigation support
(merged) #880 Use getNonAutocompleteText to fix awesomebar
(opened) #881 Reorganize Libraries and add search UI in panels
(reviewed) #882 Support using maven GV from the release channel
(merged) #883 Continue upgrading some deprecated methods
(merged) #885 Fix Language change issues when change language in Wolvic
(reviewed) #892 Use geckoview-nightly by default
(merged) #893 Migrate from org.mozilla.components:browser-search to feature-search
(reviewed) #895 Only pass valid URIs to loadUri
(reviewed) #903 Remove duplicated code from HandMeshRendererSkinned class
(merged) #906 Modernize deprecated CONNECTIVITY_ACTION in ConnectivityManager
(reviewed) #908 Fix a crash when retrieving WifiInfo
(merged) #910 Add a null check to getSignalStrength()
(reviewed) #912 Create FUNDING.yml
(merged) #913 Fix opencollective funding link
(reviewed) #914 Fix white flashes in several (heavy) WebXR experiences while in immersive mode
(merged) #917 Upgrade Android Components to 116
(reviewed) #918 [Chromium] Pass the correct URL to onPageStart
(closed) #930 Copy search engine list from upstream and add yandex when Russian
(reviewed) #935 Fix a crash when drawing hands before updating mesh in HandMeshRenderSkinned
(reviewed) #936 Revert "Modernize deprecated CONNECTIVITY_ACTION in ConnectivityManager"
(merged) #946 Fix disabling address bar auto-complete feature in settings
(merged) #947 Implement Find in Page
(merged) #950 Fix restarting Wolvic
(merged) #952 Use red color when no search result for find in page
(reviewed) #953 [Chromium] SessionFinder: fix a startup crash
(reviewed) #954 New approach to handle remote environments
(merged) #958 Open the feedback form in a new window
(merged) #959 Some minor improvements for Find in page
(merged) #960 Allow open new page without interrupting video playing
(reviewed) #643 Workaround for YouTube videos
(merged) #961 Fix Youtube captions
(merged) #966 Fix WiFi Icon when starting Wolvic with no WiFi
(closed) #968 Bypass download uri cache when user tries to download again
(merged) #969 Fix 3D Side-By-Side video playing in curved mode
(reviewed) #971 [NoAPI] Enable on screen rendering
(merged) #972 Refactor CI
(merged) #973 [NoAPI] Fix control panels
(merged) #974 Remove Find In Page Item when in kiosk mode
(reviewed) #978 Upgrade R8 to version 8.2
(reviewed) #979 [NoAPI] Enable WebXR and other fixes to the native lib
(merged) #980 Add back Find In Page when in kiosk mode
(merged) #982 Select tab when opening URL foreground by intent
(opened) #983 Use versionCodeToDate again in Settings dialog
(opened) #984 [OpenXR] OpenXRLayer move Destroy() to destructors
(reviewed) #997 Cancel find in page on navigation
(reviewed) #996 Improve detection of URLs with long gTLDs
(reviewed) #999 [OpenXR] Use XR_FB_hand_tracking_aim to get trigger pinch status and factor on Quest
(merged) #1000 Enable starting with passthrough mode
(merged) #1004 Enable to use system trusted root certificates
(reviewed) #1006 Replace Manifest's attributes instead of the whole node
(merged) #1007 Enable voice input from keyboard
(merged) #1008 Add 3D top bottom format support to VR video playing
(merged) #1009 Add 2D option to projection menu to allow exit to full screen
(merged) #1010 Move page loading progress bar around the refresh button
(opened) #1012 Enable desktop mode as the User-Agent
(merged) #1014 Make voice input content scrollable and stick to the latest
(reviewed) #1015 [HVR] Do not request the WiFi SSID for mainland China packages
(merged) #1017 Improve some UI user experience
(merged) #1018 Use horizontal layout for DateTime picker prompt
(merged) #1021 Fix several issues related to seek bar for VR video playing
(reviewed) #1020 Move homepage URL to a resValue in build config
(reviewed) #1022 [HVR] Fix flavor detection when deciding about requesting SSID
(merged) #1025 Add option to clear all user data
(merged) #1026 Add back the mute/unmute control in VR video control
(merged) #1027 Fix Youtube video pause when entering immersive mode
(merged) #1028 Allow exit find in page mode when we press the back button
(merged) #1030 Fix Chinese/Japanese keyboard typing
(merged) #1031 Use context.getCacheDir() to store unzipped environment files
(merged) #1032 Allow YouTube playing different projection types of VR videos continuously
(merged) #1033 Fix voice search default language selection
(merged) #1034 Upgrade Android Component to 121.1.0 & AGP to 8.2.1
(reviewed) #259 Do not instantiate the Runtime in crash reporter service
(reviewed) #1 Added INTERNET permissions to the manifest
(reviewed) #734 Change the window distance
(merged) #1036 Set tray date displaying format without hardcoding
(merged) #1046 Implement audio engine using Android Media Player
(merged) #1047 Allow jumping to the video start/end by clicking on time labels
(reviewed) #1049 Updated App Lab warning message
(merged) #1050 Enable drag and move windows at X & Y direction
(merged) #1057 Fix several issues related to VR videos playing
(merged) #1063 Enable seeking VR video through controller D-pad
(merged) #1064 Remove duplicated suggestions in awesome bar
(merged) #1068 Enable haptic feedback for controllers
(merged) #1071 Improve Device Name in Firefox/Mozilla Sync
(merged) #1072 Rename Firefox account into Mozilla accounts in translations
(merged) #1073 [NoAPI] Reorganize the functionalities of the buttons
(reviewed) #1076 Generalize pointer scaling and color change during trigger event
(merged) #1086 Make texture scale changeable by display DPI
(opened) #1087 Enable YouTube double captions to fit 3D video playing
(reviewed) #1091 Properly set the Quest3 device name
(merged) #1094 Add break for all cases in setting device type name
(reviewed) #1095 Remove Khronos OpenXR patch
(reviewed) #1092 Use Khronos OpenXR headers in OCULUSVR builds
(reviewed) #1099 Select 90Hz refresh rate for Quest3
(closed) #1100 Always bypass cache for some specific urls
(reviewed) #1102 Bump sharp from 0.30.5 to 0.32.6 in /tools/compressor
(merged) #1104 Enable word auto complete for Latin keyboards
(reviewed) #1105 Increase logical size and resolution of Web pages
(reviewed) #1106 Revert "Increase logical size and resolution of Web pages"
(merged) #1107 Fix widgets hovering in library UI when DPI is not 100
(reviewed) #1108 Increase logical size and resolution of Web pages (relanded)
(closed) #1111 Add head lock feature in hamburger menu
(closed) #1113 Also trigger key event when we scroll by D-pad
(merged) #1114 Fix selection menu location for web pages
(reviewed) #1116 Desktop Mode overrides for popular Chinese websites
(merged) #1119 Hide brightness button when playing video in Passthrough
(reviewed) #1124 Support head lock
(reviewed) #1085 Auto Enter WebXR
(reviewed) #1126 Fix SnapdragonSpaces build docs
(reviewed) #1125 Prevent windows out of reach
(merged) #1127 Revert "Enable showing all build warning"
(merged) #1129 Fix several issues related to "center windows vertically" and drag move in curved mode
(merged) #1130 Add group for experimental features in display settings
(merged) #1131 Download Keyboard dictionaries on demand
(merged) #1132 Add dictionaries to download in props.json
(reviewed) #1136 Different density and DPI per build
(reviewed) #1137 Bump actions/setup-java from 3 to 4
(reviewed) #1135 Add two new environments for 1.5.2
(merged) #1140 Fix disk LRU cache key formatting error
(reviewed) #1142 [ML2] Disable Hardware Acceleration for rendering UI widgets
(reviewed) #1143 Rename environment to "Winter Night"
(reviewed) #1139 [ML2] Add 3D controller model
(reviewed) #1146 Set default density to 1.25
(opened) #1147 Do not let headlock update the position of the window while resizing
(opened) #1148 Press on the skybox to reorient
(reviewed) #1152 Initialize the VR external context after initializing Java
(reviewed) #1163 Extract launch parameter names to constants
(merged) #1165 Lower the maximum display DPI to 300
(reviewed) #1169 Fix bug when moving large windows
(reviewed) #1170 [Pico] Rename PicoXR device type to Pico4x
(reviewed) #1171 Open immersive experiences directly
(reviewed) #1174 Add UA override for courses.certify-ed.com
(merged) #1175 Environments manager code logic cleanup
(reviewed) #1176 Bump com.android.tools:r8 from 8.2.33 to 8.2.42
(reviewed) #1178 Clean composing text when resetting the keyboard layout
(merged) #1185 Allow disabling Latin Keyboard input auto complete
(merged) #1189 Fix issues related to auto complete
(closed) #1192 Fix controllers disappear during video playback
(opened) #1193 Reset windows position when user tries to reorient via controllers
(merged) #1194 Improving compose text input
(reviewed) #1195 Properly support vertical videos
(reviewed) #1197 Fix controllers disappearing
(reviewed) #1198 Cleanup fullscreen code
(merged) #1200 Fix several issues when playing VR videos
(reviewed) #1182 Add support for Pico Neo3 controllers
(merged) #1205 Fix typing in Android widgets without auto compose
(opened) #1207 Remove reliance of onFirstContentfulPaint
(reviewed) #1208 Bump gradle/gradle-build-action from 2 to 3
(reviewed) #1209 Bump androidx.fragment:fragment from 1.4.1 to 1.6.2
(merged) #1210 Use doApply for setHeadLock and setWindowMovement
(reviewed) #1214 Don't go back with buttons B and Y
(reviewed) #1217 Synthesize FCP for cached pages
(reviewed) #1218 Update telemetry-related privacy options
(reviewed) #1219 [ML2] Add MagicLeap2 device type to VRControllerType
(closed) #1220 Update telemetry-related privacy options string translation
(reviewed) #1223 Modernize deprecated getMetrics(DisplayMetrics) in Display

Issues opened/helped with

(opened) #797 Modernize deprecated updateConfiguration(Configuration,DisplayMetrics) in Resources
(opened) #798 Use MediaStore.Downloads to index downloads instead
(resolved) #799 Modernize deprecated getMetrics(DisplayMetrics) in Display
(resolved) #800 Modernize deprecated setSystemUiVisibility(int) and related flags in View
(resolved) #801 Modernize deprecated AsyncTask
(resolved) #802 Modernize deprecated getConnectionInfo() in WifiInfo
(resolved) #803 Modernize deprecated CONNECTIVITY_ACTION in ConnectivityManager (and bug fix on new methods)
(opened) #805 Modernize deprecated JobIntentService
(resolved) #810 Remove deprecated cookieLifetime
(resolved) #822 Modernize deprecated dispatchConfigurationChanged(Configuration)
(resolved) #823 Upgrade the CMakeList.txt
(resolved) #835 Missing some symbols / keys from the virtual keyboard
(resolved) #841 Migrate from org.mozilla.components:browser-search to feature-search
(resolved) #214 Firefox Accounts authentication breaks if you navigate to another page
(resolved) #845 Keyboard layout doesn't match the system's language
(resolved) #777 Need to format file sizes consistently
(resolved) #653 Use a different application name for HVR mainland China package
(addressed) #89 [OpenXR] When headset tracking is off, windows appear at "ground" level (Oculus)
(resolved) #778 The Download Confirmation Dialog shouldn't center the question
(resolved) #423 Implement date/time picker
(resolved) #654 Target Android API level 32
(resolved) #688 Keep the selected options of HTML <select> multiple Attribute
(resolved) #875 Dark mode support
(resolved) #715 Remove WaveVR build dependency
(resolved) #890 Local Gecko builds are not used by default when building a package
(resolved) #896 DateTime picker prompt dialog is not properly triggered
(closed) #922 Add Yandex to the list of available search engines
(closed) #928 Missing the "Share with other apps" option in the downloads lists
(resolved) #923 Youtube VR videos with lower quality than expected
(resolved) #929 Search in page
(resolved) #943 The 'Address bar auto-complete' feature can not be disabled
(addressed) #939 Tabs get closed with updates
(resolved) #949 "RESTART NOW" does not restart
(resolved) #940 Implement the feedback form with a native UI dialog
(resolved) #717 Youtube Captions
(resolved) #964 False positive network status when starting Wolvic
(resolved) #874 No playback in Apple TV streaming service
(resolved) #712 Too much restart needed to restablish unstable connection
(resolved) #666 Persistent setting for the passthrough mode
(addressed) #493 Increase window parameter adjustment options in resize menu
(addressed) #353 Download dialog cannot show repeatedly
(addressed) #482 Empty canvas element after exiting VR
(addressed) #662 Leak of surface when exiting VR mode
(resolved) #545 3D SBS not working in curved mode
(resolved) #489 [Privacy and Security] Add trusted Root CA option
(resolved) #461 Add more options to view 3D-SBS-Videos
(addressed) #927 Drag and rotate windows using the controller
(resolved) #619 When running a noapi build, a black screen is displayed
(resolved) #970 Some Web XR Links do not work in Kiosk Mode
(addressed) #977 Restore the FxR inherited versionCode auto generation
(resolved) #981 Opening a URL from the command line always opens a new tab
(addressed) #976 [OpenXR] Revamp finalization of OpenXRLayer subclasses
(resolved) #998 Date picker can be too tall
(addressed) #86 Rendering artifacts in Atomic City scene of Mozilla Hubs
(addressed) #225 No audio casting stream from Wolvic
(resolved) #226 No sound effects playing Moon Rider (https://moonrider.xyz)
(addressed) #782 Support KTX v2
(resolved) #660 Speech input on vr keyboard
(resolved) #171 Fragmented subtitles in 3D side-by-side movie
(addressed) #498 Needs to start in desktop mode
(resolved) #299 in kiosk mode, maybe it needs a loading progress bar of the webxr content
(addressed) #87 Saving and loading don't work in Brushworkvr app
(addressed) #1011 Dot (.) unconditionally appended to voice search text
(resolved) #1013 Voice input content UI overflow and not stick to the latest
(resolved) #963 Environments download should use getFilesDir()/getCacheDir() to store the downloaded zip file
(addressed) #598 Media stream detached from window in some cases?
(addressed) #731 💡 The 360° background could be updated according the webpage
(resolved) #345 on a shared device clearing cache and history it does not reset everything
(resolved) #236 softlocking quest when opening dll file in external app
(resolved) #1023 Evaluate the need for the media session extension
(addressed) #473 Add a glTF/glb loader
(addressed) #1029 Make the CrashReporterService work again
(resolved) #1044 Implement an Android's MediaPlayer AudioEngine
(addressed) #99 Click and Drag windows
(resolved) #1055 There is no curved window in full-screen video mode
(resolved) #1058 华为vr glass
(resolved) #1059 华为vrglass
(resolved) #1062 Not installable on Quest 3
(resolved) #1067 Enable haptic feedback when controller pointer swipes across widgets/web page
(resolved) #1069 Allow 'Device Name' change in Firefox/Mozzila Sync settings.
(addressed) #1065 can wolvic support WebGPU? and if not, what's the roadmap?
(resolved) #737 Request: autocorrect/text prediction
(resolved) #1078 How to improve sharpness at 0.5 window size
(resolved) #1093 Allow users to use the Zoom to change the size of the rendered web content
(addressed) #1097 CORS errors on same origin inconsistent with other browsers (Same Origin Policy)
(resolved) #544 InputMethodManager: Display ID mismatch found
(resolved) #1088 Default values for display density and DPI
(addressed) #1110 Spatial navigation support for AR/VR controllers with D-pads
(opened) #1121 Investigate the feasibility of newer Mozilla Android components for sessions
(resolved) #1128 Download keyboard auto-complete dictionaries on demand
(resolved) #938 Zoom in and out
(resolved) #1133 Pico 4 upload a new environment?(adb command not recognize wolvic * file/folder)
(resolved) #1134 Request to adjust screen distance and increase resolution for pico4
(resolved) #1164 Setting DPI too high causes thermal runaway
(resolved) #1166 Disabling 360 background
(resolved) #1162 wolvic support to play 360 view & Horizontal Panoramic View pictures which is photoed by the customers
(addressed) #1183 Text added to the wrong place when autocomplete is active
(resolved) #1181 Controllers disappear during video playback
(resolved) #1186 Autocomplete: blank space requires tapping on the spacebar twice
(resolved) #1187 Autocomplete: keyboard becomes sluggish when autocompleting medium-long words
(addressed) #1211 Automatic VR 360 WebXR Open

Code reviews and Suggestions from SARIF report

Hollow Man — Tue, 16 May 2023 19:31:38 +0000

What I built

SARIF support for Reviewdog

Category Submission:

Maintainer Must-Haves

App Link

https://github.com/HollowMan6/sarif4reviewdog/pull/5

Screenshots

Description

SARIF, the Static Analysis Results Interchange Format, is a standard, JSON-based format for the output of static analysis tools.

This action enables the maintainers to integrate the fixes proposed by static code analysis tools right from the PR comments.

Link to Source Code

https://github.com/HollowMan6/sarif4reviewdog

Permissive License

MIT

Background

https://github.com/orgs/community/discussions/52156

GitHub Code Scanning supports generating alerts from the SARIF report, but it can't show the proposed fixes from the report. It can be a good addition to give us the option to make commits to the codebase based on the proposed fixes.

Then I submitted the feature request, but no one respond anything, so why not realize it by myself instead?

How I built it

In SARIF, The fix object represents a proposed fix for the problem indicated by the Result. It specifies a set of artifacts to modify. For each artifact, it specifies regions to remove and provides new content to insert.

I build a general converter from SARIF to Reviewdog Diagnostic Format (RDFormat), then use Reviewdog to give suggested code changes as well as the context of the changes for PR reviewing.

Additional Resources/Info

This Action is currently used by kubescape, a Cloud Native Computing Foundation (CNCF) sandbox project. It is an open-source Kubernetes security platform and includes risk analysis, security compliance, and misconfiguration scanning. Targeted at the DevSecOps practitioner or platform engineer, it offers an easy-to-use CLI interface, flexible output formats, and automated scanning capabilities.

My CNCF LFX Mentorship Spring 2023 Project at Kubescape

Hollow Man — Sun, 14 May 2023 12:49:20 +0000

https://www.youtube.com/watch?v=ZBK5wKpvM3s&t=1773s

Project Link: CNCF - Kubescape: Release engineering: add Kubescape to commonly-requested package managers

kubescape is a Cloud Native Computing Foundation (CNCF) sandbox project. It is an open-source Kubernetes security platform and includes risk analysis, security compliance, and misconfiguration scanning. Targeted at the DevSecOps practitioner or platform engineer, it offers an easy-to-use CLI interface, flexible output formats, and automated scanning capabilities.

List of things I have done

Documentations:

Publishing

Repo and Packages created:

PRs opened:

(merged) kubescape/kubescape #1095 fix(build): LICENSE file in release tarballs
(merged) kubescape/kubescape #1105 fix(README): broken links
(merged) kubescape/kubescape #1140 ci(release): fix publishing krew plugin; add '.exe' extension to Windows binary
(merged) kubescape/kubescape #1147 Change installation path to ~/.kubescape/bin
(merged) kubescape/kubescape #1148 arm64 release binaries for CI and Krew
(merged) kubescape/kubescape #1169 Add kubescape.exe to the release assets
(merged) kubescape/kubescape #1184 feat(sarif): add fix object in generated reports
(merged) kubescape/kubescape #1185 fix(fix): mixed up change summary list
(merged) kubescape/kubescape #1186 Invoke packaging workflow to update after release
(merged) kubescape/kubescape #1196 Move building instructions to wiki, add more installation instructions
(merged) kubescape/kubescape #1199 Update installation script
(merged) kubescape/kubescape #1210 ci: update before install packages
(merged) kubescape/kubescape #1213 Deprecate kubescape-windows-latest
(merged) kubescape/kubescape #1214 Add ref to workflow dispatch
(merged) kubescape/kubescape #1216 Make powershell Windows installation user path available immediately
(merged) kubescape/kubescape #1236 Deprecate kubescape-windows-latest and fix CI
(merged) kubescape/kubescape #1238 Fix downloading arm64 binary for kubescape
(merged) kubescape/github-action #32 Support for code reviews instead with PRs
(merged) kubescape/github-action #34 Fix start new PR with own repo
(merged) kubescape/github-action #37 Add exceptions parameters back
(merged) kubescape/github-action #38 Keep kubescape github-action workflow up to date
(merged) kubescape/github-action #41 Intergrate fix with comments
(merged) kubescape/github-action #42 Version bumps start PRs instead of committing directly
(merged) kubescape/github-action #43 Split the suggest fix workflow and update docs
(merged) kubescape/vscode-kubescape #11 Remove platformPackages config
(merged) kubescape/vscode-kubescape #12 Bump kubescape version into v2.3.1
(merged) kubescape/lens-extension #16 Bump kubescape version into v2.3.1
(merged) kubescape/node-kubescape #3 Support for ARM64 binaries as well as kubescape.exe
(merged) kubescape/homebrew-tap #7 Add Auto Release CI
(rejected) kubernetes-sigs/controller-runtime #2266 🐛 Support get config inside snap with SNAP_REAL_HOME
(rejected) kubernetes/kubernetes #117165 client-go: support detect homedir with SNAP_REAL_HOME and os/user.HomeDir
(merged) ScoopInstaller/Main #4757 kubescape: Update url and binary naming
(pending) gentoo/gentoo #30595 sys-cluster/kubescape: new package, add 2.3.3
(merged) chocolatey-community/chocolatey-packages #2226 (kubescape) Add Kubescape package

Issues opened/helped with:

(resolved) kubescape/kubescape #195 Provide ARM64 release binaries
(resolved) kubescape/kubescape #400 Add Kubescape to packages management for easier installation
(reviewed) kubescape/kubescape #720 Error Fixed when downloading on azure cloud vm environment
(reviewed) kubescape/kubescape #1014 Package manager support: homebrew
(resolved) kubescape/kubescape #1015 kubescape installed in first directory in $PATH under $HOME
(helped) kubescape/kubescape #1033 Generate SLSA provenance for builds
(reviewed) kubescape/kubescape #1112 can't install Kubescape with krew on Apple Silicon Mac
(resolved) kubescape/kubescape #1142 Package manager support: RPM
(resolved) kubescape/kubescape #1143 Package manager support: deb
(resolved) kubescape/kubescape #1168 Add kubescape.exe to the release assets to replace kubescape-windows-latest
(resolved) kubescape/kubescape #1183 Add fix object in Kubescape generated SARIF reports when available
(reviewed) kubescape/kubescape #1215 Fix issue 11552
(helped) kubescape/kubescape #1237 Download and Installing Wrong Binary For Apple M1
(reviewed) kubescape/kubescape #1239 Added instructions to setup kubescape locally
(resolved) kubescape/k8s-interface #46 Kubescape supports getting packed as a snap app
(resolved) kubescape/k8s-interface #46 Kubescape supports getting packed as a snap app
(pending response) github/community #52156 Support code auto-fixes for GitHub Code Scanning
(resolved) chocolatey-community #2186 Migrate kubescape package
(resolved) chocolatey-community/chocolatey-packages #2190 (kubescape) Migrate package
(resolved) snapcraft/store-requests #34661 Request for classic confinement and name change for cli-kubescape
(pending response) snapcraft/snapd #34683 Feature Request: Stop using $SNAP_REAL_HOME to visit real home files

Project summaries

Packaging

Other packages managers that have already been available and not introduced by me during this project period:

OpenSUSE Zypper
Homebrew
Krew (I added the ARM64 support for Krew)
Nix-pkgs
Scoop

GitHub Actions Release CI

I helped improve the Kubescape GitHub Actions release CI process, where I added the ARM64 build and tested for the GitHub Actions release CI workflow. I use QEMU with Docker to simulate the Linux ARM64 environment for building and testing the binaries. For macOS M1/M2, I investigated how to cross-build libgit2 C code and use Golang cross-compilation to build the binaries.

I also helped add the auto version bumping CI for kubescape/homebrew-tap, kubescape/packaging, and kubescape/github-action. After the release is made, we trigger these CIs so that the kubescape versions in these repositories can get upgraded automatically.

GitHub Actions Code Review

I helped improve the Kubescape GitHub Actions fix suggestions code review process, where I created the workflow which works by collecting the SARIF (Static Analysis Results Interchange Format) file that kubescape generates. Then, with the help of HollowMan6/sarif4reviewdog, convert the SARIF file into RDFormat (Reviewdog Diagnostic Format) and generate reviews for code fix suggestions on GitHub Actions using Reviewdog. I also helped add the “fix" object support for the Kubescape-generated SARIF report.

In addition to the main project, I also helped the community with other issues like bug-fixing as well as feature-adding.

Microsoft Reinforcement Learning Open Source Fest 2022 – Native CSV Parser

Hollow Man — Sat, 13 Aug 2022 15:20:00 +0000

Reinforcement Learning (RL) Open Source Fest 2022 Final Project Presentations

Shorter Introduction

PR: https://github.com/VowpalWabbit/vowpal_wabbit/pull/4073

Tutorial: https://vowpalwabbit.org/docs/vowpal_wabbit/python/latest/tutorials/cmd_csv_with_iris_dataset.html

Introduction

My project here at the Reinforcement Learning Open Source Fest 2022 is to add the native CSV parsing feature for the Vowpal Wabbit.

So why do I choose to implement native CSV parsing? CSV is one of the most popular file formats used in the machine learning dataset and is often delivered as the default format in competitions such as Kaggle. CSV files have the same schema for each example, while VW text format doesn't. Although converters in Python and Perl have been written which convert these files to VW text format, it would be convenient if VW could understand CSV files natively. I also want to challenge myself, as there is a surprising amount of complexity in the design of implementing a generalized parser for CSV, so this project is as much about considering all of the design pieces as implementing a working parser. My choice and time devoted also pay off since my project has already got merged into the VW upstream !!!

About CSV

CSV files are often separated by commas (,) or tabs. however, alternative delimiter-separated files are often given a .csv extension despite the use of a non-comma field separator. This loose terminology can cause problems in data exchange. Many applications that accept CSV files have options to select the delimiter character and the quotation character. Semicolons (;) are often used instead of commas in many European locales in order to use the comma (,) as the decimal separator and, possibly, the period (.) as a decimal grouping character.

Separating fields with the field separator is CSV's foundation, but commas in the data have to be handled specially.

How do we handle CSV files?

The short answer is that we follow the RFC 4180 and MIME standards.

The 2005 technical standard RFC 4180 formalizes the CSV file format and defines the MIME type "text/csv" for the handling of text-based fields. However, the interpretation of the text of each field is still application-specific. Files that follow the RFC 4180 standard can simplify CSV exchange and should be widely portable. Here are its requirements:

Lines will end with CR or CRLF characters, and it is optional for the last line.
CSV files can have an optional header record. There is no sure way to detect whether it is present, so care is required when importing.
Each record should contain the same number of separated fields.
Any field may be quoted with double quotes.
Fields containing a double-quote or commas should be quoted.
If double-quotes are used to enclose fields, then a double-quote in a field must be represented by two double-quote characters.

Dig into details

Allows specifying the CSV field separator by --csv_separator, default is ,, but " | or : are reserved and not allowed to use, since the double quote (") is for escape, vertical bar(|) for separating the namespace and feature names, : can be used in labels.
For each separated field, auto remove the outer double-quotes of a cell when it pairs. --csv_separator symbols that appeared inside the double-quoted cells are not considered as a separator but a normal string character.
Double-quotes that appear at the start and end of the cell will be considered to enclose fields. Other quotes that appear elsewhere and out of the enclose fields will have no special meaning. (This is also how Microsoft Excel parses.)
If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote, and will remove that escape symbol during parsing.
Use header line for feature names (and possibly namespaces) / specify label and tag using _label and _tag by default. For each separated field in header except for tag and label, it may contain namespace and feature name separated by namespace separator, vertical bar(|).
--csv_header to override the CSV header by providing the header. Combined with --csv_no_file_header, we assume that there is no header in the CSV file and under such condition specifying --csv_header for the header is a must.
If the number of the separated fields for current parsing line is greater than the header, an error will be thrown.
Trim the field for ASCII "white space"(\r\n\f\v) as well as some UTF-8 BOM characters(\xef\xbb\xbf) before separation.
If no namespace is separated, will use empty namespace.
Separator supports using \t to represent tabs. Otherwise, if assigning more than one character, an error will be thrown.
Directly read the label as string, interpret it using the VW text label parser.
Will try to judge if the feature values are float or string, if NaN, will consider it as a string. quoted numbers are always considered as strings.
If the feature value is empty, will skip that feature.
Reset the parser when EOF of a file is met (for possible multiple input file support).
Support using --csv_ns_value to scale the namespace values by specifying the float ratio. e.g. --csv_ns_value=a:0.5,b:0.3,:8, which the namespace a has a ratio of 0.5, b of 0.3, empty namespace of 8, other namespaces of 1.
If all the cells in a line are empty, then consider it an empty line. CSV is not a good fit for the multiline format, as evidenced by many empty fields. The multiline format often means different lines have different schemas. However, I still leave the empty line support to ensure that it's flexible and extendable. In this case, users can still express multiline examples in CSV files, although it is not listed as supported. We still throw an error if the number of fields separated by the line doesn't match the previous, even if all the fields are empty, as this usually means typos that users may not intend.

Some statistics to share

The project reaches 100% Test and Code Coverage. On my computer, it takes only 829 ms to handle 200000 examples, which is comparable to the equivalent VW custom format data file’s parsing speed of 623 ms, and is equivalent to 21.7 MB/s throughput.

I have expected that little bit of slower between VW custom text format and the CSV format. The VW text parser can parse as it reads since generally the elements have a fixed position, while CSV parser needs to store the split elements into an array, and look up that array according to the header for labels, tags, namespace and feature values, also there's double quotes trimming and escape support, which will definitely cost more time.

After all, I have tried my best to optimize it and the performance is also to my satisfaction. You can check using the throughput tool for the unoptimized one during the project. And that was actually 10 times improvement!

VW text Parser:

2239418 bytes parsed in 60581μs
36.9657MB/s

Unoptimized CSV Parser:

1799450 bytes parsed in 912927μs
1.97108MB/s

Optimized CSV Parser:

1799450 bytes parsed in 87728μs
20.5117MB/s

My optimization mainly involves turn all the functions used by the parser into force inline. During the parsing, instead of parsing column by column, I categorize all the features by its namespace and parse those features by namespace to avoid double caculating the namespace hash. I also replace the map with unordered map, use vw's own data structure v_array to replace std::vector, and use VW's realization of parsefloat to replace std::stof, which is much faster. The statistics show that these really improved the performance a lot.

Credits

That's all, thanks for listening, and also special thanks to my mentors Jack Gerrits and Peter Chang from Microsoft Research - New York City Lab, who really helped me a lot during the project.

Reference

Comma-separated values - Wikipedia

Tencent Rhino-bird Open-source Training Program 2022 – SunEC sm2p256v1 Key Pairs Generation

Hollow Man — Sat, 13 Aug 2022 14:56:00 +0000

For 2022 final result, I passed the final evaluation, obtained all the honor titles (Only 2/4000+ in all participants): Tencent Open Source Contributor Certificate (only issued 30+ around the globe at that time), Intern of Excellence, Task Scholarship.

Issue

Task 1

Requirement

https://github.com/openjdk/jdk/pull/9541

Task 2

Requirement

Result

Tested computing the signature as well as verifying the signature for comparing between secp256r1 and secp256k1 using SHA256withECDSA with the help of the SunEC provider.

The result clearly shows that secp256r1 has a better performance than secp256k1 with regard to SHA256withECDSA when signing. But secp256r1 has almost the same performance as secp256k1 when verifying.

Benchmark                                 Mode  Cnt     Score    Error  Units
BenchmarkSigning.secp256k1_1024B         thrpt   25  1237.482 ± 30.518  ops/s
BenchmarkSigning.secp256k1_1024K         thrpt   25   318.589 ±  2.656  ops/s
BenchmarkSigning.secp256k1_128B          thrpt   25  1266.561 ± 32.311  ops/s
BenchmarkSigning.secp256k1_256B          thrpt   25  1254.327 ± 36.935  ops/s
BenchmarkSigning.secp256r1_1024B         thrpt   25  1746.453 ± 33.358  ops/s
BenchmarkSigning.secp256r1_1024K         thrpt   25   340.530 ±  3.970  ops/s
BenchmarkSigning.secp256r1_128B          thrpt   25  1763.460 ± 31.179  ops/s
BenchmarkSigning.secp256r1_256B          thrpt   25  1756.899 ± 32.715  ops/s
BenchmarkVerifying.secp256k1_1024B       thrpt   25   486.545 ± 13.410  ops/s
BenchmarkVerifying.secp256k1_1024K       thrpt   25   228.638 ±  2.478  ops/s
BenchmarkVerifying.secp256k1_128B        thrpt   25   491.065 ± 13.948  ops/s
BenchmarkVerifying.secp256k1_256B        thrpt   25   499.466 ±  4.558  ops/s
BenchmarkVerifying.secp256r1_1024B       thrpt   25   402.902 ± 53.489  ops/s
BenchmarkVerifying.secp256r1_1024K       thrpt   25   212.743 ± 23.301  ops/s
BenchmarkVerifying.secp256r1_128B        thrpt   25   401.215 ± 65.401  ops/s
BenchmarkVerifying.secp256r1_256B        thrpt   25   393.021 ± 79.755  ops/s

Further investigation shows that before secp256k1 was removed from JDK, all the curves seem to be realized by C using the OS library instead of Java. https://github.com/openjdk/jdk/blob/jdk-11+28/src/jdk.crypto.ec/share/native/libsunec/impl/oid.c#L95

JDK-8238911 in 2020 reported the weaknesses in the implementation of the native library EC code make it necessary to remove support for future releases. The most common EC curves (secp256r1, secp384r1, and secp521r1) had been re-implemented in Java in the SunEC JCE provider.

After some communications with my mentor Johns Jiang, he tells me that JDK-8181594 introduces the optimized finite field implementations in Java. Previously before that implementation was introduced, pure Java realization was really slow, then we use the OS library to realize all the curves so that the performance can be improved. But now, instead, with the help of that optimized Java library, Java realization takes the advantage and becomes the most efficient one, it's now even comparable with the pure C realization.

Our flame graph also some kind confirms this, as you can see here, the Java Flight Recorder (JFR) can record the secp256r1 methods calling stacks, but it's not the case for secp256k1. So it's likely that secp256r1 has a better performance than secp256k1 for signing since it's fully realized in Java and using that optimized library, thus reduces the calling costs for the OS library. If they are both realized in Java using the optimized method, I guess there should be no difference.

As secp256k1 has already been removed in JDK and now secp256r1 does have a better performance, so I guess here we will have no obvious further room for improvement.

Task 3

Requirement

The final report (in Chinese) is not limited to the following content, it also includes some more security analysis for different ways of realization.

With modification of SunEC provider

Result

As for Elliptic-curve based cryptography algorithms, the curve parameters are used to generate the keys.
The official recommended curve parameters for SM2 can be seen here:
https://www.oscca.gov.cn/sca/xxgk/2010-12/17/1002386/files/b965ce832cc34bc191cb1cde446b860d.pdf

That curve parameters is also known as sm2p256v1, since it's also a prime curve just like secp256r1, we can use the existing implementation of secp256r1 in the SunEC library to help us realize our implementation.

The OID for sm2p256v1 is 1.2.156.10197.1.301: http://gmssl.org/docs/oid.html

https://github.com/HollowMan6/jdk/tree/sm2
https://github.com/HollowMan6/jdk/commit/c3e924641bb3a838f6abc496dd380ceb619df163

We first fill the curve parameters into the CurveDB

Then add the OID and names.

The most important part is FieldGen. FieldGen is used to automatically generate optimized finite field implementations, which is also the library I mentioned in Task 2 JDK-8181594 for improving Java version's efficiency. https://github.com/HollowMan6/jdk/blob/c3e924641bb3a838f6abc496dd380ceb619df163/make/jdk/src/classes/build/tools/intpoly/FieldGen.java We need to generate two fields, Integer Polynomial (corresponds to parameter p) and Order Field (corresponds to parameter n).

As:

FFFFFFFE FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 00000000 FFFFFFFF FFFFFFFF

FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF $2^{256} - 1$

00000001 00000000 00000000 00000000 00000000 00000000 00000000
00000000 $2^{224}$

00000000 00000000 00000000 00000000 00000000 FFFFFFFF FFFFFFFF FFFFFFFF
$2^{96} - 1$

00000000 00000000 00000000 00000000 00000000 00000000 FFFFFFFF FFFFFFFF

$2^{64} - 1$

= $2^{256} - 2^{224} - 2^{96} + 2^{64} - 2^0$

So the Integer Polynomial shall fill just like that. We can copy other parameters from secp256r1 as they are all 256 digit prime curve.

The private keys in Hex can be printed directly with no special format.

Since the public keys in Hex can be compressed, it does have a special format, that if it starts with 04, then the keys are uncompressed and we just then concat the X and Y coordinate together. The compressed ones always start with 02 or 03, and then only the X coordinate is needed. The 02 and 03 is determined by that, when Y coordinate is even, we use 02, use 03 when odd. In addition, we have to also make sure that both X and Y coordinates are 64 in length for Hex.

As the Bouncy Castle library has already fully realized the SM2, to ensure that the generated keys fit the sm2p256v1, I also use the generated keys for signing using SM3withSM2. The validity of the keys can be verified during the signature verification processes, during which we recover the sm2p256v1 elliptic curve point from the Bouncy Castle library based on the Hex format public key. If we use keys generated based on other curves, like secp256r1, error will be thrown.

Demo result:

Public Key (Uncompressed): 040A8FB35CA4761FAAA36B2A24E77EC657D96F74147C50EE2D5B50E3AAFD8304D8CBB65FB2E661D37B7C3B900E1BDBEDE894D9CBB9079E8DD704B9465BFF65EE17
Public Key: 030A8FB35CA4761FAAA36B2A24E77EC657D96F74147C50EE2D5B50E3AAFD8304D8
Private Key: 42756D22960A58D08F9E7E3A0D56D9630D8D051D082F4D2BFCE22FD2653524EB
To sign: How are you?
Signed: MEUCIAgvYgl0ydHwd536MkmwaRuhmkD/klh79VmHEBJI1zCRAiEAo9jNkGM+Tjh/0AmX82nSPOMYgRPaWm6SUXiB63YGAD4=
Verification OK!

Error thrown when using secp256r1 (faulty key pairs):

Public Key (Uncompressed): 0456A3205C7B47BF303F4B65CDAB5B94C343BE7220E5AAAB001B7263DBCD113B42447A9E41BF1374D4ABC7A2AE31E7441E3EB20D5808CCB7D88BFE4F8F2C9887C3
Public Key: 0356A3205C7B47BF303F4B65CDAB5B94C343BE7220E5AAAB001B7263DBCD113B42
Private Key: A33048CC39925B5D4BA4C34FE846C85D41DA5AABA0CFDE4092A7A4ED716D557
To sign: How are you?
Signed: MEYCIQCktncEmfLbC9rLC1Im9gj/AvZRUIQ5Z1plrq1X0L/5YwIhAOa0JeSoQFnV51kAJsFRY3T4cpCn2O7bKoN+M+nPpv6y
Exception in thread "main" java.lang.IllegalArgumentException: Invalid point coordinates
        at org.bouncycastle.math.ec.ECCurve.validatePoint(Unknown Source)
        at org.bouncycastle.math.ec.ECCurve.decodePoint(Unknown Source)
        at org.sample.SM2Util.verify(SM2Util.java:103)
        at org.sample.SM2Util.main(SM2Util.java:129)

JMH Performance test for compressed and uncompressed public key generation result shows that the compressed public keys generation has a better performance than the uncompressed ones. You can find the reason from the flame graph of Java Flight Recorder (JFR), that it's because the uncompressed ones also need to caculate the Y coordinate Hex format, which takes a lot of time.

Benchmark                                         Mode  Cnt       Score       Error  Units
BenchmarkPublicKeys.sm2p256v1_compressed         thrpt   25  1212201.531 ± 248181.084  ops/s
BenchmarkPublicKeys.sm2p256v1_uncompressed       thrpt   25   760033.805 ±  35058.515  ops/s

Homemade

Our realization refers to the JavaScript implementation here, Wikipedia Elliptic curve point multiplication and the official documentation .

The homemade one is based on purely mathematical methods, no other dependencies.

We use SecureRandom to generate the private key, we also tried public key generation using the Standard Projective Coordinates and Jacobian Coordinates, as well as the binary expansion and addminus algorithm.

See result for more performance comparisons.

JMH Performance test for compressed and uncompressed public key generation result shows that the uncompressed public keys generation has almost the same performance as the compressed ones. Though it seems like a contradiction, here the uncompressed Y coordinate Hex caculation counts significantly smaller when you take the overall time into consideration.

Benchmark                                         Mode  Cnt       Score       Error  Units
BenchmarkPublicKeys.sm2p256v1_compressed         thrpt   25  786.038 ± 12.099  ops/s
BenchmarkPublicKeys.sm2p256v1_uncompressed       thrpt   25  795.960 ±  8.557  ops/s

My Google Summer of Code 2022 – Google Blockly Workspace MultiSelect Plugin

Hollow Man — Sat, 13 Aug 2022 14:44:58 +0000

My Presentation at 2023 Blockly Developer Summit: Multi-Select Plugin

More about 2023 Blockly Summit

Project Repository Link

Try out the plugin online

PRs and issues resolved during GSoC 2022:

My GitHub

Mailing list for tracking

Hi there! If you prefer watching videos, check this out for an episode demoing the plugin: https://www.youtube.com/watch?v=FZyvvPZhIRs

For you guys who prefer reading, in this blog post I will introduce you to my work on the plugin for selecting, dragging, and doing actions on multiple blocks. The project is sponsored by Google Summer of Code (GSoC) 2022, under MIT App inventor.

Backgrounds

Aim

The project aims to enable the selection of multiple blocks at the same time and allow moving and doing actions on multiple blocks on Blockly.

This behavior is the same as when you try to manage your files on your Operating System. You can click on the files while pressing the control key, drag a rectangle to select multiple files. Then move them around, copying and deleting.

It sounds a little bit easy, but actually, I would say that it's not. Multiple selections can become a crazy-complex feature when you start thinking about the details.

History

This feature request has remained open on GitHub Issues for six years. However, it was still in the discussion phase and far from the beginning of Implementation before my project began.

Since the Blockly community long wants this feature, we base our plugin on the latest Blockly so that it can be applied to everyone's project. The App Inventor uses a Blockly version that is much older, so it's a pity that we can't see it work on App Inventor now. Let's hope that the App Inventor can upgrade the Blockly version to the latest soon.

Realization

The "drag a rectangle to select" feature is realized with the help of the DragSelect plugin. I submitted a PR to add the pointer-events so that it can work for Blockly, and it got merged into v2.4.4.

In addition, I disable the drag surface feature in Blockly, which stops us from moving multiple blocks simultaneously. Also, there's evidence suggesting that we can perform better without a drag surface.

So, how does the plugin work? Well, generally, the plugin acts like an adapter. It maintains its own multiple selection set, which keeps currently selected blocks and make sure we always have one of the selected blocks as the selected one in Blockly core. When users do some actions, the plugin also passes all the actions to the other blocks in our set besides the selected one in Blockly core.

Functionalities

Let's check out what the plugin can do!

Additional blocks can be selected by holding the SHIFT key while clicking the new block. You can also deselect the block by clicking the already selected block.
Clicking on the button above the trashcan is equivalent to holding or releasing the SHIFT key for switching between the multiple selection mode and the normal mode.
We can clear the selection by clicking on the workspace background.
Clicking a new block without holding SHIFT key can clear the multiple selections and change the selection to only that block.
Holding SHIFT key to drag a rectangle area to select can reverse their selection state for the blocks touched by the rectangle.
In multiple selection mode, workspace dragging and block dragging will all be disabled. You can only drag to draw a rectangle for selection.
When some of the selected blocks are in one block stack, for example, some top blocks and some of their children blocks are in the selection simultaneously. If applicable, the plugin only disconnects the selected most top block in that stack with its parent block. Move along with all the children's blocks of that most top block as a whole.
You can also drag all the blocks to the trash can.
When you edit the fields while selecting multiple blocks, we will automatically apply that to all the blocks with the same type.
There's also an MIT App Inventor-only feature that has been migrated into this plugin, that you can double click to collapse or expand currently selected blocks.
For the context menu, the Duplicate will duplicate the selected most top block in the block stack and all the children blocks of that most top block. The selection will be changed to all newly created duplicate blocks' most top blocks. For all the other items, The actions to show are determined by the state of the block which the user right-clicks on, and the same action will be applied to all the blocks no matter their individual state. We will append the currently applicable number of user-selected state-changing blocks, and the number will only be shown when it is greater than 1.
The Add Comment / Remove Comment option will add / remove comment buttons to all the selected blocks.
The Inline Inputs / External Inputs option will convert the input format with all the selected blocks.
The Collapse Block / Expand Block option will only apply to the selected most top block in the block stack.
The Disable Block / Enable Block option will only apply to the selected most top block in the block stack. All the children blocks of that most top block will also get disabled.
The number in Delete [X] Blocks is the count of the selected most top block in the block stack as well as all children of those selected most top block. Clicking on that option will delete the blocks mentioned.
The Help option displays just the helping information of the block the user just right-clicked on.
We add Select all Blocks in the workspace context menu.
For the shortcut keys, These actions will only apply to the selected most top block in the block stack. when you press Ctrl A, you can select all the blocks in the current workspace. Ctrl C to copy the selected blocks, Ctrl X to cut the selected blocks to the clipboard, and Ctrl V to paste all the blocks currently in the clipboard and get all the newly pasted blocks selected.
Bumping neighbors after dragging to avoid overlapping is disabled in this plugin by default, since I find it disturbing sometimes for multiple selections.
Click on a block will bring that block to the front in case Bumping neighbours is disabled.

Usage

If you want to integrate the plugin into your project, you can add the dependency into your package.json. In the source code, pass the workspace to the plugin, and initialize the plugin, and then it's done! You can choose to disable double-click the blocks to collapse or expand and enable bumping neighbors after dragging to avoid overlapping. For the multi-select controls, you can also choose to hide the icon and customize the icons of each state. More details can be found in the README.

You can also choose to integrate the plugin with other ones, like the scroll options, which enable the edge scroll and wheel scroll for Blockly. The only thing you have to pay attention for the scroll options plugin to work is to assign the original blockDragger value required for scroll options to baseBlockDragger in our plugin. During the project period I also submitted a PR for fixing the a bug that makes scroll options unable to work without the drag surface, and it has already got merged.

Finally

That's all for this blog post. Before it ends, I would like to say thank you to my mentors, Evan Patton and Li Li, as well as the MIT App Inventor Team for guiding me throughout the project. They are really supportive. Also, special Thanks to Beka Westberg. She devoted a lot of time to giving suggestions and helping review the code. We can't have this plugin without her! Finally, thanks for reading this blog post! If you have any questions, please comment, and I'll reply. Cheers!

My Summer of Bitcoin 2022 Project - CI for CADR

Hollow Man — Sat, 13 Aug 2022 14:03:00 +0000

Synopsis

Before the Summer of Bitcoin project, Cryptoanarchy Debian Repo (CADR) lacked Continuous Integration (CI), which troubles the new coming contributors because setting up the developing environment can be complex. I finally successfully implemented the CI using GitHub Actions default runners. The CI can be triggered manually, or by sending PRs as well as pushing directly to the master branch.

The CI is divided into 2 jobs:

The first one is the build job. It builds the running podman environment image, uploads the image to Artifacts for reuse. Then with the podman environment, builds the CADR packages, checks their sha256 sum, and then uploads the built Debian packages as well as their sha256 sum values to Artifacts.
The second one is the test job. It runs when the build job finishes. The testing jobs are run in parallel for each package. The test job first downloads the built images and packages Artifacts uploaded in the build job, then use make test-here-basic-% and make test-here-upgrade-%(% for package name) to run tests.

Road of Implementation

First try

At first, I ignored the fact that there already exists a dockerfile for CADR running (although it was not for building), and setup my own dockerfile from scratch by adding dependencies when I encounter any errors.

My own dockerfile turned out to work fine on GitHub Actions for the building process, but failed for the test process.
Initially I thought it was some more dependency issues, since the test can work on my own computer.

When checking the logs I found that it's due to the unshare issue.

Then I noticed that adding a --privileged parameter can fix the unshare issue for docker. But then a systemd issue just came after it. Finally I noticed the dockerfile in the codebase that already exists for CADR, and just as how it works, I made the systemd to start as the first process.

VOLUME [ "/sys/fs/cgroup" ]
CMD ["/lib/systemd/systemd"]

and add --tmpfs /tmp --tmpfs /run --tmpfs /run/lock -v /sys/fs/cgroup:/sys/fs/cgroup:ro as additional parameters to share some host machine resources and make it as the daemon container. Operations are done using docker exec to attach to the container. But still it doesn’t fix the systemd issue.

Finally I switched to podman and the systemd issue got fixed (it supports --systemd=true) because I occasionally found this article when I Googled the issue.

However, a new issue occurs suggesting failed to override dbcache.

Can't see any errors from the workflow logs, even though the missing bc dependency has been fixed and I can confirm that the command here works correctly in the container running on my PC. Moreover when I manually skip the test just mentioned, the test further shows that there is an unneeded reindex:

Which suggests that it's related to issue 108 maybe? Have opened an issue here. I can skip that test, but further error suggesting electrs is not available. It can be fixed with this PR

Even if those issues are fixed, another platform may still be needed since the full test requires too much space:

Only testing the regtest part will be fine with no space issue.

Fixing the tests

The first issue is to find out why overriding dbcache fails. This takes me quite a few weeks to find out the solution. Originally, when determining bitcoin dbcache size, bc is invoked. But bc is not guaranteed to be installed, thus it can fail. Then instead of doing maths wizardry, I submitted a PR to just match on ranges: RAM < 1024 -> default dbcache, RAM < 2048 -> dbcache=512, ..., RAM => 8192 -> dbcache=4096. However, the failed to override dbcache still exists when running in GitHub Actions workflow with this PR. It's very weird, as when I run the test manually using the same methods from the workflow with the podman container on my PC, such an issue won't appear. So the issue seems to belong to GitHub Actions workflow environment only even though it's running in the container.

Then after a lot of trial and error, I finally found the culprit. By referring to PR, previously the debcrafter dropped all capabilities which seems to cause errors when the host kernel capabilities do not match those known to setpriv. We need to only drop supported capabilities by the current kernel.

Then I submitted a PR to fix this, and got merged.

The second issue is to find out why the unneeded reindex error exists. I noticed that although the test failed on test-here-basic-electrs with unneeded reindex when running make test, running make test-here-basic-electrs alone won't fail. So prior to fixing this issue, my guess was that maybe it's still related to issue issue 108, which is that after updating existing non-pruned nodes in experimental -reindex was used despite pruning not being changed. Or the test environment didn't get cleaned up before test-here-basic-electrs when executing make test. My final result proves that the later is right, and submitted a PR to clean up the chain mode marker, since we want it to be clean with package_clean_install.sh

I also submitted a PR to force linked-hash-map version to be 0.5.4 for fixing the build issue of debcrafter that just came up during the project period.

The test can be successful on GitHub Actions when only tested with regtest and after PR 200, 201, 203, 204 get merged.

Research on cloud provider

I also researched on which cloud service to use, as we were intended to use a GItHub self-hosted runner. As I checked on cloud service providers AWS, Google Cloud and Azure. I find that GitHub Actions uses Azure as their default runners, to build a GitHub self-hosted runner, Azure would be a good choice if we use the same provider as the GitHub default hosted runner. Also Azure has a trial period with 200 dollars for one month when starting a new account, so we can start free, while other cloud service providers don’t offer such a great discount.

Investigation on other possible CI platforms

I find out that unlike GitHub Actions that runs CI in a virtual machine, Travis CI, GitLab CI/CD, Jenkins all run the CI predefined Docker containers without without systemd support. Then it would be not possible to run a systemd supported podman container inside such a container. Azure DevsOps is a valid one for our use case, and I also tried to use it. Then I noticed that for a single CI job, each step, it only allows 1 hour maximum, otherwise it would get cancelled, while our build job as well as full net test job last much longer than 1 hour, it's also not suitable for us. Then finally the GitHub Actions would be the only good choice to have.

Self-hosted runners for GitHub Actions

I had also tried to use Azure Virtual Machines service by myself to set up Self-hosted runners for GitHub Actions, but it seems like the environment doesn't automatically get cleaned each time, and is contaminated from previous runs.

Divide and Conquer

Finally I came up with a new way for testing. We can use the divide and conquer to bypass the short of disk space issue as well as the contaminated running environment issue when running tests on both the mainnet and regtest. I can see that the make test is composed of make test-here-basic-% and make test-here-upgrade-%(% for package name), we can just use the matrix in GitHub Actions to test each package in separate environments, and then the disk space would be enough, and PR 204 can be closed because we now always have a clean environment.

Then I successfully implemented and tested that method, and it works! So right now it's the full test and I have reached the project goal with even a solution that has no money cost for CI building and testing!

The CI can be successful if the previous tests fixing PRs get merged first.

Final Deliverables

After my mentor @Kixunil's review, I began to use --locked parameter to make cargo use cargo.lock which contains correct versions. Also I fixed a security issue, make a user account user and built/tested with user, and made to upload the sha256sum result for the built deb packages afterwards so people can check hashes.

In addition, I fixed tests for bitcoin-regtest after setting bitcoind nosettings enabled in PR 205. Then there should be some kind of bug in core for this wallet location difference in the GitHub runner and the physical machine, so the PR gets closed and another solution that makes tests independent of wallet location was committed.

The CI runs successfully with the above changes, and everything works fine, nothing left to do.

Conclusion

My Summer of Bitcoin project is a great experience for me as I learned a lot about Bitcoin and related DevOps knowledge.

If you are interested in Bitcoin and would like to start contributing to Cryptoanarchy Debian, with my work, you can easily fork the repo and add more test or new packages by committing the code to your fork's master branch on GitHub, the CI will help you build the deb packages and locate possible errors, no need to setup the developing environment on your computer again.

Summary of PRs

Merged

Closed as resolved in another way

Schedule notifications on WeChat with Azure AKS

Hollow Man — Sun, 06 Mar 2022 06:35:50 +0000

Overview of My Submission

An application that allows you to schedule notifications on WeChat so that you can get notified while keep connected with your friends.

This app is realized through several WeChat message push platforms, such as PushPlus and ServerChan, as well as the native Enterprise version of WeChat (WeCom) message push API.

The application be deployed using Azure AKS by helm chat with this script. For meanings of configurations please refer to the README file.

Submission Category: Computing Captains

Link to Code on GitHub

HollowMan6 / Wechat-Timed-Message

Send timed message to Wechat. (向微信推送定时消息)

向微信推送定时消息

(English version is down below)

源Github项目地址：https://github.com/HollowMan6/Wechat-Timed-Message

好用记得收藏(右上角加星★Star)哦!

微信消息推送脚本

工作流存放文件夹

Docker镜像支持在amd64、arm64、arm32v7、arm32v6、ppc64le、s390x、i386、mips64le多架构(包括大型机、普通PC机以及树莓派等开发板)运行。

使用方法

你需要fork本仓库，之后在你fork的仓库中创建相关Actions Secret并进行相关设置(按下图所示点击1，2，3的次序，即可进入新建Actions secrets的界面):

你可以从以下三个推送平台中任选一个或多个来接受推送的消息：

PushPlus(推荐)

登录PushPlus，然后在pushplus网站中找到您的token，创建一个Name为PPTOKEN，value为您的token值的Actions secret，就可以进行一对一推送信息。

如果需要对多个账号推送信息，即一对多推送，还需要另外新建一个群组，记下群组编码，然后创建一个Name为PPTOPIC，value为您的群组编码的Actions secret。

企业微信推送

首先注册企业微信。
注册成功后，获取企业id，创建一个Name为CORPID，value为您的企业id值的Actions secret。
选择应用管理 → 应用 → 创建应用
自己创建完成后获取Secret和AgentId，创建分别Name为CORPSECRET和AGENTID，value为这些值的Actions secret。
进入我的企业 → 微信插件，拉到下边扫描二维码，关注以后即可收到推送的消息。

效果示意示例内容

Server酱

如果要使用Server酱，请创建一个/修改Name为SERVERCHANSCKEY，value为你的SendKey值的Actions secret。

上述配置成功后，配置工作流文件，以工作流1.yml为模板，创建你自己的工作流或者在提供的工作流上进行修改。你可以任意更改name为无空格的英文字母和数字组合的字符串，cron为你想要发送消息的指定时间(你可以使用crontab guru进行cron表达式的调试，所有时间均为UTC时间，请进行时区换算)(因为Github方的原因，预定运行时间可能会有半小时左右的延迟)。然后创建一个或者两个Actions secrets，一个必须创建，其name为TITLE[name]（请将这里的[name]修改为workflow的name），value为要发送消息的标题，例如在提供的工作流中，这里的name为TITLE1；另一个为可选的，其name为MSG[name]，同理进行相应的替换，value为要发送消息的标题。

随后，按下图所示点击1，2，3，4的次序，你可以手动触发工作流的执行来进行测试。

点开任意一个运行记录，依次点开下图所示1，2，你可以看到运行记录。

如果某次因为某些因素工作流运行失败，GitHub会自动发邮件提醒工作流运行失败。

新：增加可选的遇到发送消息失败的情况，自动重启工作流，并等待一段时间后再次发送消息。如果你需要这个功能，则请创建一个Personal Access Token, 获取教程(第7步令牌的作用域权限你只需要选中workflow这一栏即可)。然后创建一个Name为GPATOKEN，value为你的令牌值的Actions Secret。

默认再次发送消息等待时间为30分钟，如果你有需要可以修改你的fork仓库对应的这里的30m替换为你想要的数值，这里的时间遵循Linux sleep 函数对应时间语法：一个数字后接 s 对应秒, m 对应分钟等。

如果是因为本仓库程序本身因为失效而导致的报错，你可以取消正在运行中的工作流从而终止这一循环。

GitLab

GitLab仓库地址：https://gitlab.com/HollowMan6/Wechat-Timed-Message

你还可以使用GitLab的Pipeline来进行消息推送：

同理，fork完GitLab仓库到你的名下后，创建Variables（类似于GitHub Actions Secrets）：

同理，你还可以创建一个Pipeline triggers(类似于Github Access Token)并加入到Variables中来启用自动打开失败再次消息推送：

默认再次消息推送等待时间为30分钟，如果你有需要可以修改你的fork仓库对应的…

View on GitHub

Additional Resources / Info

https://docs.microsoft.com/en-us/azure/aks/quickstart-helm#run-your-helm-chart

openEuler 20.03 LTS SP3 Kickstart Automatic Installation on VMware Workstation

Hollow Man — Thu, 17 Feb 2022 13:59:07 +0000

Introduction

As openEuler also uses Redhat Anaconda as its OS installer, we can use Kickstart for Automatic Installation. However, the guides provided in the previous documentation officially are not straightforward enough for Linux newbies who tend to use virtual machines for installation. So in this blog, I'm going to show you how to perform openEuler 20.03 LTS SP3 Kickstart Automatic Installation on VMware Workstation with only a single ISO file referring to the CentOS Kickstart Installations Documentation.

Steps

Create the Kickstart file

Copy the following code into a file named ks.cfg at somewhere on your computer.

#version=DEVEL
# Use graphical install
graphical


%packages
@^minimal-environment
@standard

%end

# Keyboard layouts
keyboard --xlayouts='cn'
# System language
lang zh_CN.UTF-8

# Network information
network  --bootproto=dhcp --device=ens160 --onboot=off --ipv6=auto --activate
network  --hostname=localhost.localdomain

# Use CDROM installation media
cdrom

# Run the Setup Agent on first boot
firstboot --enable
# System services
services --enabled="chronyd"

ignoredisk --only-use=nvme0n1
autopart
# Partition clearing information
clearpart --none --initlabel

# System timezone
timezone Asia/Shanghai --utc

# Root password
rootpw openeuler
user --groups=wheel --name=openeuler --password=openeuler --gecos="openEuler"

%addon com_redhat_kdump --disable --reserve-mb='128'

%end

%anaconda
pwpolicy root --minlen=8 --minquality=1 --strict --nochanges --notempty
pwpolicy user --minlen=8 --minquality=1 --strict --nochanges --emptyok
pwpolicy luks --minlen=8 --minquality=1 --strict --nochanges --notempty
%end

The above Kickstart file will set Chinese as the default language, and set the root password to openeuler, with an additional account name and password both openeuler.

If you want to create your own Kickstart file, you can firstly perform a manual installation you would like, after successfully installing and logging into the system, get the generated Kickstart file that corresponds to the manual installation just performed at /root/anaconda-ks.cfg, rename it to ks.cfg.

Generate the ISO file

If you are on Linux, run mkisofs for ISO generation:

cd <the folder where you store your ks.cfg>
mkisofs -JR -V OEMDRV -o auto-installation.iso ks.cfg

If you are on Windows, you can use UltraISO for ISO generation:
open UltraISO, click on the New button, rename the disk into OEMDRV.
Press F3 to add file ks.cfg into the ISO, then click on the Save button to save the ISO file as auto-installation.iso.

Configure the Virtual Machine

Please first create the virtual machine on VMware Workstation for openEuler 20.03 LTS SP3. If you don't know how to do this, please refer to the VMware Workstation documentation or related blogs as there're already many tutorials that exist.

In the Virtual Machine's tab, click on the Edit virtual machine settings.
On the prompted Virtual Machine Settings window, click on the Add... button.
On the prompted Add Hardware Wizard dialogue, choose CD/DVD Drive, click on the Finish button.
Then for the newly added CD/DVD Device, choose Use ISO image file:, and click on the Browse... button to select the ISO file auto-installation.iso just created in the last step, finally click on the OK button.

At last, start your virtual machine and enjoy Kickstart automatic installation!

mdbook-pdf: A mdBook backend for generating PDF files

Hollow Man — Sun, 30 Jan 2022 07:09:48 +0000

Introduction

mdBook allows you to create book from markdown files. It's pretty much alike Gitbook but implemented in Rust. However, unlike Gitbook that supports using calibre for generating PDF, for a long time, mdBook doesn't support generating PDF files natively, and supporting that is also not in their roadmap. Existing plugins (backends) such as mdbook-latex that utilize Tectonic as well as pandoc solutions will generate a PDF page that doesn't unify with the existing mdBook generated HTML version. Considering these facts, I created a mdBook backend named mdbook-pdf for generating PDF based on headless chrome and Chrome DevTools Protocol Page.printToPDF.

mdbook-pdf depends on Google Chrome / Microsoft Edge / Chromium. The generated page are pretty much alike the one you manually print to PDF in your browser by opening print.html or executing google-chrome-stable --headless --print-to-pdf=output.pdf file:///path/to/print.html, but with customization of PDF paper orientation, scale of the webpage rendering, paper width and height, page margins, generated PDF page ranges, whether to display header and footer as well as customize their formats, and more, as well as automation. It supports all the platform where Google Chrome / Microsoft Edge / Chromium would work. You can check samples of the generated PDF files in the Artifacts here (The Rust book collections generated in x86_64 Windows, macOS as well as Linux).

Installation & Usage

Since it's a plugin (backend) for mdBook, first of all you should ensure that mdbook is available.

If your machine's architecture is x86_64, or you are using Linux for ARM64, check the successful build GitHub Actions workflows, click into the latest one, and then you can get a binary from the Artifacts (including Windows, Linux, macOS).

Otherwise, make sure the rust compiling environment is available, execute cargo install mdbook-pdf to compile and install.

If you want to compile the latest version, make sure the Rust build environment is available (cargo build).
Then run git clone https://github.com/HollowMan6/mdbook-pdf.git, in the cloned folder, run cargo build --release , get the executable in target/release/, and put it in PATH.

For running, have Google Chrome / Chromium / Microsoft Edge available (installed at the default location, in PATH or binary location configured) as currently, automatically downloading Chromium binary isn't available (will update once upstream fixes such support).

On Windows 10 and above, the program can generate PDF normally without installing any additional software, because Microsoft Edge is the browser provided with Windows system. Of course, considering the support for the older versions of Windows without Edge, you can install Google Chrome on your computer.
In MacOS, you need to install Google Chrome / Microsoft Edge or Chromium.
In Linux, you can choose to install any of the Google Chrome / Chromium / Microsoft Edge browsers. It is recommended to install Chromium. The name of this software package in your Linux distribution is commonly chromium or chromium-browser (Note: for Ubuntu later than 18.04, you have to install chromium-browser through snap).

Make sure the following exists in your book.toml:



[output.html]

[output.pdf]

And also [output.html.print] is not disabled (it should be enabled by default, so don't worry if the following lines doesn't occur in you book.toml).



[output.html.print]
enable = true

A simplest book.toml is as follows:



[book]
title = "An Example"

[output.html]

[output.pdf]

Finally you can build your book and get the PDF file with mdbook build command, your PDF file will be available at book/pdf/output.pdf.

Configuration

Check book.toml and comments for details for the available configurations of [output.pdf].

Credits

This project relies on headless_chrome. Because the new version has not been released, and the default timeout is not friendly to PDF generation, I use my Fork version to publish mdbook-pdf-headless_chrome for expanding the relevant timeout to 300 seconds as a submodule of this project, thus enabling the project to be published on Crates.io as well.

Some Notes and Thoughts

mdBook supports adding alternative backend. When the mdbook build command is invoked, if the book.toml in the book folder has an [output.pdf] item in addition to the default [output.html] for generating the html web page, mdbook-pdf will be called, and the relevant book information and parameter configuration in JSON are passed to the standard input of the program. Relevant mdBook documentation can be found here.

A headless browser means that all the operations will be in the background without a graphical interface.

Before decided to use Chrome DevTools Protocol Page.printToPDF, I also tried wkhtmltopdf which is based on QT4 Webkit. Hovewer, wkhtmltopdf doesn't seem to support CSS Printing @media Rule, which will make some extra components visible and printed in the PDF.

I've also tried directly making HTTP calls to the WebDriver in python based on W3C WebDriver Protocol Print Page, as well as using selenium for calling Chrome DevTools Protocol Page.printToPDF in Python. All those methods are not robust for large pages with the errors as follows for just the same reason for the original upstream headless_chrome: the default timeout is 10 seconds, which is not friendly to large page PDF generation.



timeout: Timed out receiving message from renderer: 10.000



selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed

At present, I have tested that Firefox does not support using headless_chrome for PDF generation, and the Safari browser even does not support the W3C WebDriver Protocol Print Page, let alone Chrome DevTools Protocol.

I've noticed that for some book, by using this backend, when click on some links that link inside the book, a html page that points to the original generated HTML storing path of the book will be opened, just as the issue mentioned here. I guess for those "internal" links inside the book, work should be done in the mdbook side for print.html referring here so that all the links linked "internally" would jump inside the generated print.html, as all the contents should already be on the print.html, there shouldn't be any hyperlinks that jump to other html files in the book. By resolving in this way, the generated PDF would also jump internally instead of opening a browser that won't connect to anything. I have already created a PR for this, hope it will get merged soon.

In addition, as I found that special characters such as : are very likely to appear in the title of the book, which will cause the generation of the related PDF file to fail, so the file name does not use the form of <book name>.pdf but output.pdf

Hope you enjoy it.

Deploy a Kubernetes Cluster based on Calico and openSUSE Kubic

Hollow Man — Thu, 27 Jan 2022 16:37:17 +0000

Introduction

openSUSE Kubic is a certified Kubernetes Distribution based on openSUSE MicroOS. Calico is an open-source project that can be used by Kubernetes to deploy a pod network to the cluster. In this blog, I will show you how to deploy a Kubernetes Cluster based on Calico and openSUSE Kubic by a Virtual Machine. We are going to deploy a cluster that has a master and a worker.

I was intended to use Oracle VM VirtualBox. However, it turned out that on my machine, when I tried to run kubeadm at openSUSE Kubic in VirtualBox, it always stuck at watchdog: BUG: soft lockup - CPU#? stuck for xxs! with CPU usage around 100%. As a result, I switched to VMware Workstation Pro and the issue got solved. Guess it's caused by some bugs of VirtualBox.

Steps

Create the Virtual Machine and Install openSUSE Kubic

Here I won't explain how to do these things but share some important things to note, just refer to their documents if you don't know or have any questions.

Here is my configuration for the Virtual Machine. Recommend that your host machine has a memory that is larger than 8GB so that more than 3GB of memory can be assigned to the Virtual Machine for it to run smoothly. In order for the Virtual Machines to be connected to each other, and also connect to the Internet, you can set the Network Adapter to be Bridged (Automatic).

For openSUSE Kubic Installation, remember to choose kubeadm Node when it comes to System Role, as it will deploy a Weave pod network cluster instead of Calico if you choose to use Kubic.

I suggest that you can install the openSUSE Kubic in one Virtual Machine, later after successful installation, clone that Virtual Machine, assign one as master and another worker. Remember to do a full clone.

Configuring the Master

When you boot into the master Virtual Machine, you can see your IP address in the notification part. In my case, it's 192.168.1.14. Take note of that.

For the convenience of copy and paste commands, we can use SSH to log into the system. To configure that, first, log into the system with root account. Second, execute vi /etc/ssh/sshd_config.d/10-enable-root-password.conf, type i to insert, write the following into the file:

PasswordAuthentication yes
PermitRootLogin yes

This will enable SSH root password login, although it's not recommended if you are in production. When editing is finished, press [ESC] then type :wq to save and exit.

Kubeadm Init

Run kubeadm config images pull to pull the container images required for Kubernetes.

You can also specify the --image-repository if, in your location, registry.opensuse.org downloading speed is too slow. In my case (in China) I'll use Aliyun to speed up: kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers

Then run kubeadm init --apiserver-advertise-address=<Your Master IP Address> --pod-network-cidr=192.168.0.0/16, replace <Your Master IP Address> with the IP address you just noted. If you specified the --image-repository in the last step, also append that to this command.

Wait for it to finish, remember to take notes of the worker nodes joining command.

Execute export KUBECONFIG=/etc/kubernetes/admin.conf in your shell for the kubectl to work.

Deploy Calico

Get the latest copy of the calico configuration yaml file by curl -O https://docs.projectcalico.org/manifests/calico.yaml.

Change the path to install the FlexVolume driver by sed -i 's#/usr/libexec/kubernetes/kubelet-plugins/volume/exec#/var/lib/kubelet/volume-plugin#g' calico.yaml as in Transactional (Atomic) systems /usr/libexec/kubernetes is read-only.

Finally, apply the yaml file by kubectl apply -f calico.yaml.

Wait for all the pods to be available.

watch kubectl get pods --all-namespaces

You can check the Events of the pod to get the error messages if you are waiting too much time on a specific pod: kubectl describe pods -n kube-system <Pod Name>.

Configuring the Worker

Start the worker Virtual Machine, login as root, change the host name as it can't be the same with the master: hostnamectl set-hostname 'worker'.

Finally, execute the worker nodes joining command just noted, ignoring the hostname could not be reached warnings since we didn't and don't need to configure the DNS.

Then wait for the worker to be available, Done!

GitHub Issues to Wechat

Hollow Man — Tue, 30 Nov 2021 15:09:56 +0000

My Workflow

HollowMan6 / GitHub-Issues-to-Wechat

Send GitHub Issues, PRs or Discussions Updates to Wechat.(将 GitHub Issues, PRs 或者 Discussions 更新消息发送到微信)

Send timed message to Wechat

(中文版本在下方)

Send GitHub Issues, PRs or Discussions Updates to Wechat

Source Github Repository Link: https://github.com/HollowMan6/GitHub-Issues-to-Wechat

Please ★Star if you think it's great!

Python library dependency

Source Code

Example Workflow

Usage

you can fork this repository first, and then create Actions Secrets and set related settings in your forked repository (click in the order of 1, 2 and 3 as shown in the figure below).

You can choose one or more of the following three push platforms to receive pushed messages:

PushPlus(Recommended)

First log into pushplus, and then find your token in pushplus website, create a actions secret with the name of PPTOKEN and the value of your token value, and then one-to-one push the related information results.

If you need to push the related information to multiple Wechat accounts, that is, one-to-many push, you need to create a group, write down the group code…

View on GitHub

Submission Category:

Maintainer Must-Haves

Yaml File or Link to Code

https://github.com/HollowMan6/GitHub-Issues-to-Wechat/blob/main/.github/workflows/message.yml

Additional Resources / Info