DEV Community: Azka

Setting Up Github Pages using Jupyter Book

Azka — Tue, 08 Mar 2022 11:16:37 +0000

(This tutorial is cross-posted on my github.)

Jupyter Book vs Jekyll for Github Pages

Both will generate static site for you, which means you can write your pages in markdown and they will built your html pages based on the layout settings.

(+) Native support for jupyter notebook file (ipynb). While you can put your ipynb file to Jekyll (see the example from my page here), it needs to be converted to html first before Jekyll converts it to another html with the predetermined layout settings.

(+) Easier to install since you only need to install jupyter-book and ghp-import packages in your python environment. Less headache for Windows users since you don't need to install Ruby and/or Linux distribution.

(-) No customized template available for Jupyter Book. Jekyll has plenty of themes (check them here).

If you still wish to build Github Pages using Jekyll, check my tutorial and my page.

Live Version

Check the live version of my Jupyter Book here.

Jupyter Book Tutorials

Assumptions

You are already familiar with:

git
command line

Steps

The official tutorial from Jupyter Book covers pretty much all of the required steps. I added some notes based on my experience:

Install Jupyter Book

Starts from this page. Make a new environment, solely for making jupyter book and install the package there. I recommend to use Python 3.7.

However, there is a known incompatibility for notebook execution when using Python 3.8.
Tweak Some Files

Follow the instructions from the beginning until the "Add it to your Table of Contents" part in "Create your own content file" page. I repeatedly encountered build failure which coming from Github Actions trying to build the references section in the later step. Remove references.bib and comment out the line where it includes references in _config.yml file. Proceed to the next steps, starting from "Re-build your book".
Prepare gh-pages branch

When you finally get to "Publish your book online with GitHub Pages" part, you will need to setup gh-pages branch (step 2). By now, your jupyter book is already available in your repository but not yet deployed as a page. Open your repository in browser (i.e. https://github.com/azukacchi/notebooks) then create a branch called gh-pages, which will contain all files as your main branch. You should have two branches (main and gh-pages) in your repository. Finish the remaining steps.

Note that you don't need to make changes to the gh-pages branch, only the main branch. Any changes made to the main branch will be applied by ghp-import package by overwriting all files in gh-pages. The gh-pages is solely used as the source for your Github Pages.

Notes

When you want to update your book, you should always make your changes to the main branch:

git add ./*
git commit -m "update xxyyzz"
git push

Then rebuild your book, i.e. for my notebooks repository:

jupyter-book build notebooks/
ghp-import -n -p -f notebooks/_build/html

Setting Up Github Pages site with Jekyll Tutorial

Azka — Mon, 22 Mar 2021 07:23:41 +0000

This tutorial is cross-posted on my github

Jekyll is a static site generator. With Jekyll, you can write your pages in markdown from which it will build your html pages based on the layout settings. For example, if the markdown file you're writing is project1.markdown, Jekyll will build the project1.html file for you. There's no need to tweak the html, css, or whatever it is for the layout unless you want to do some customization. The Jekyll theme you're using alone already allowed some customization (think of working with Bootstrap) and you can easily change them using the provided field.

Live Version

Check the live version of my portfolio https://azukacchi.github.io/

Assumptions

You are already familiar with git
You have the project documentation ready in markdown

Tutorial Links

Steps

Install Ruby and Jekyll

Follow the instruction from this page for Windows.

Prepare Your Project Folder

We will prepare and test out pages locally first. These first steps are modified from this.

Make the new project folder. Choose any names, e.g. myblog, because this won't be the name for your repository, as your default github page is username.github.io.
Open command prompt and navigate to project folder.
Choose the directory where you are going to publish your site.
- If you choose to publish your site from the root of the repository, /, your github pages will appear directly on the root, e.g. username.github.io. If you choose this, you don't need to make any new directory
- If you choose to publish your site from a specific folder of the repository, e.g. /docs, your github pages will appear on that page, e.g. username.github.io/docs. If you choose this, make a new directory, then navigate to that folder.
```
mkdir docs
# Creates a new folder called docs
cd docs
```
Create a new Jekyll site in the current directory
```
jekyll new
```
Install your theme, e.g. "Just-the-Docs" theme. The default theme for new Jekyll site is "minima".
```
gem install just-the-docs
```
add it to your Jekyll site’s Gemfile
```
gem "just-the-docs"
```
Add Just the Docs theme to your Jekyll site’s _config.yml
```
theme: "just-the-docs"
```
Run your local Jekyll server then open your local site on browser: http://localhost:4000
```
bundle exec jekyll serve
```

Make Changes to Your Pages

In this step, you can try adding some pages, customization, and test them locally. When you're changing the _config.yml file, the update will not be applied unless you restart the server and run this line again: bundle exec jekyll serve

Always check the theme documentation to write your pages and setting them up! When you want to customize your theme CSS, check the theme documentation.

After you've completed setting up your pages and your site is running okay locally, proceed to the next step.

Push Existing Project to Github

Create a new repository on Github. Type a name for your repository. If you're creating a user site, your repository must be named <yourusername>.github.io. Do not add any files because we're going to push the files from local.
Unless you're already working in the root of your project folder, navigate to the root. Initialize git repository in the current directory (must be the root folder).
```
git init
```
Edit the Gemfile that Jekyll created.
- Add "#" to the beginning of the line that starts with gem "jekyll" to comment out this line.
- Add the github-pages gem by editing the line starting with # gem "github-pages". Change this line to:
```
gem "github-pages", "~> GITHUB-PAGES-VERSION", group: :jekyll_plugins
```
  Replace GITHUB-PAGES-VERSION with the latest supported version of the github-pages gem. Check the version here: "Dependency versions".
- Save and close the Gemfile
If you're using Jekyll theme other than the supported themes, edit your _config.yml file. For example, for "Just-the-Docs" theme, change this line: theme: "just-the-docs to this:
```
remote_theme: pmarsceill/just-the-docs
```
For another theme, check the theme documentation.
From the command line, run bundle update

Add your GitHub repository as a remote

git remote add origin https://github.com/<username>/<username>.github.io.git

Then add, commit, and push your files to the remote

git add .
git commit -m "initial commit"
git remote -v
git push origin master

Open your github pages. If there's something wrong, e.g. theme doesn't load, check your email! When there's something wrong, you will get a Page build failure or Page build waning email.

Use a Customized CSS

If you're using a customized CSS (e.g. changing or adding new styles) and run them locally, you usually edit the theme by editing a scss file in the directory of the theme. For example, for "Just-the-Docs" theme, your custom.scss file is in this directory:

C:\Ruby27-x64\lib\ruby\gems\2.7.0\gems\just-the-docs-0.3.3\_sass\custom\custom.scss

Check the path to the installed theme directory using this bundle info --path "THEME-NAME"

However, when you're running your page on Github Pages, you must make the directory and the file yourself in the folder where you will publish your site. The exact directory is different for each theme, e.g. for "Just-the-Docs", the directory for the new file is _sass/custom/custom.scss. You should follow your theme documentation when you add the contents of the new file. Example with Minima theme here.

Deploy Flask Application to PythonAnywhere or Heroku (without database)

Azka — Sat, 13 Mar 2021 05:21:46 +0000

language: Bahasa Indonesia

Contoh project: http://nyc311-azka.herokuapp.com/

Source code: github

PythonAnywhere (paling mudah)

tutorial: https://help.pythonanywhere.com/pages/Flask/

Deploy di PYAW paling mudah karena tidak membutuhkan kita untuk menginstall aplikasi tertentu di komputer, tidak seperti Heroku yang membutuhkan aplikasi Heroku dan Git.

Catatan:

Kalau aplikasinya tidak perlu memakai package dengan versi tertentu, saran saya tidak perlu membuat virtualenv karena memori yang disediakan sangat terbatas (500 MB). Ketika menggunakan virtualenv memori akan termakan banyak untuk install package.
File yang dibutuhkan untuk project kita tinggal di-upload ke PYAW. Jika project terdiri dari beberapa folder dan file (seperti project pada umumnya), akan lebih mudah jika satu kesatuan folder project kita (atau folder tertentu, misal hanya folder static) di-zip terlebih dahulu lalu di-upload di lokasi yang diinginkan.
Contoh: untuk project nyc311 yang membutuhkan file static dalam folder static, upload folder static yang sudah di-zip ke direktori /home/azukacchi/nyc311. File zip dapat di-unzip dengan cara: (1) membuka folder tempat file zip berada lalu "Open Bash console here", atau (2) buka bash console dimana pun lalu ubah direktori ke tempat file zip berada cd /home/azukacchi/nyc311. Setelah bash console terbuka, ketik unzip namafile.zip (dalam kasus ini namafile adalah static). File zip yang terekstrak akan menjadi folder berisi file-file.
Jangan lupa isi detail aplikasi di halaman Web.

Heroku

diadaptasi dari tutorial resmi: Getting Started on Heroku with Python

Asumsi:

Heroku dan Git sudah ter-install (jika belum, lihat link tutorial)
folder project sudah tersedia dengan (dengan atau tanpa menggunakan virtual environment), aplikasi Flask sudah berjalan OK di komputer dan siap di-deploy

Langkah-langkah:

Di dalam folder project, buat beberapa file dengan menggunakan Notepad bernama Procfile, requirements.txt, runtime.txt, .gitignore dengan rincian isi file sebagai berikut:
1. Procfile
```
  web: gunicorn app:app
```
2. requirements.txt
  
  Semua package yang dibutuhkan untuk project kita. Tambah juga gunicorn untuk webserver (install dengan pip terlebih dahulu). Contoh untuk project saya
```
  gunicorn
  flask==1.1.2
  imblearn
  plotly==4.14.3
  pandas==1.2.2
  numpy==1.19.2
  sklearn
```
3. runtime.txt
  
  Isi dengan versi python yang digunakan untuk project. Misal untuk project ini saya menggunakan virtual environment dengan python versi 3.8.5, maka isi file sebagai berikut.
```
  python-3.8.5
```
4. .gitignore
  
  Sesuaikan dengan project. Misal:
```
  ## Cache
  __pycache__/
```
Dengan asumsi seluruh file sudah final, langkah selanjutnya adalah menjadikan folder project kita menjadi repository git. Buka command prompt lalu buka direktori folder project kita dan ubah folder menjadi repository git. Lalu buat aplikasi heroku baru di folder tersebut. Contoh:
```
cd "C:\Users\azuka\Drive Folder\Dash\311-nyc"
git init
heroku create
```
Commit dan push. Perhatikan baik-baik log pada command prompt jika terdapat error pada langkah terakhir. Error bisa datang dari versi python yang tidak sesuai di file runtime.txt atau ada package yang tidak tertera di file requirements.txt.
```
git add .
git commit -m "lalalayeyeye isi bebas"
git push heroku master
```
Jika deploy sudah berhasil, pastikan satu instance app berjalan
```
heroku ps:scale web=1
```
Ganti nama app lama (secara default di-generate random oleh Heroku) dengan nama yang diinginkan
```
heroku apps:rename newname
```
Jika kita membuat perubahan pada aplikasi, cukup ulangi deploy dengan mengulangi langkah nomor 3.

Making a simple twitter autobase bot that posts incoming DM

Azka — Tue, 08 Sep 2020 17:27:10 +0000

Script can be found here: https://github.com/azukacchi/twitter_autobase

Description

(Very) simple twitter autobase bot that posts incoming DM

Features

1. Posts incoming DM that contains the trigger word.

2. Posts incoming DM with media. Three ways of sending a post containing media through DM:

Upload your own image with the text you want to add. However, DM will not be sent if it contains uploaded attachment other than photos (example: GIF/video). The result is something like this.
Embed images/gif/video from an existing tweet by copy-pasting the link contained in a tweet with images/gif/video. Example: to embed the image from this tweet, copy the text and take only the link.

The text would be something like this:

190831 인천 스카이 페스티벌 #NCTDREAM #NCT #해찬 #HAECHAN https://pic.twitter.com/r97xH3s15X

Take only the link and paste it into the DM along with the text you want to add and the trigger word like below.

your text hereeeeee lorem ipsum triggerword
https://pic.twitter.com/r97xH3s15

The result is something like this.

To submit a quote-RT of a tweet, simply insert the link of the tweet like below.

this is going to be sent as a quote-RT your text hereeeee triggerword https://twitter.com/timeto_haechan/status/1300093500810170370

The result is something like this.

3. Cuts post longer than 280 characters and posts them as a thread. The result is something like this.

4. Sends a DM back for each incoming DM.

status sent: DM was successfully posted, the link to the posted tweet will be attached.
status notsent: DM was not posted because it does not contain the trigger word.
status wrong attachment: DM was not posted because it contains uploaded attachment other than photo. #### 5. Deletes the incoming DMs once they are posted. You can still see the content of the original DM from the link in the notification message you send back to the sender. Optional: you can also delete the notification DM you've just sent by uncommenting this line in senddm method.

#api.destroy_direct_message(int(notifdm.id))

6. Makes a log for each run into `test.log` file.

What do you need

Make sure you have already installed Python.
A Twitter developer account; sign in to your Twitter account then apply for a developer account here. The time required until your application is approved might varies (mine was 10-14 days 😬). After your application is approved, make sure to change the permission to the "Read, Write, and Direct Messages". Generate the keys and tokens after you change the permission setting.

How to run this

If you have git installed, clone this in your directory. If not, just download the ZIP file.

  git clone https://github.com/azukacchi/twitter_autobase.git

Install the required packages (or check requirements.txt to install each package separately).

  pip install -r requirements.txt

Set the required fields in config.py:
- Put the keys and tokens you get from your Twitter developer account.
- Set your timezone and the trigger word you want to use.
Modify the notification message as you wish in the senddm() method.

How do I automate the script

You can use Task Scheduler on Windows (open Start then search Task Scheduler) to run the app.py. This app is quite simple to use, the only downturn is it only runs when your computer is either turned on or sleep but not shutdown (obviously). Some helpful tutorials can be found here and here.

Notes

This bot is not suitable for a big autobase account. The checkdm() method only checks the DM to your bot account, without saving the tweets to a dabatase. Then this bot will post the incoming DM as soon as it runs the post_all() method, with a certain time span between the tweets to avoid Twitter API rate limits. To run a big autobase account with higher posts rate, coming from higher followers counts, such as @collegemenfess with 500K+ followers, you'll need to check the dm more often and you'll need a database that enables you to store the incoming DM and post them later.

Questions

I've just started this as a new hobby during pandemic, if you have any questions feel free to reach me on Twitter ahah ☺️

Blocking twitter trolls (2): Blocking fancams

Azka — Mon, 24 Aug 2020 13:15:14 +0000

Earlier this year, a kpop translation account started running a script that blocks all trolls who post fancam under their tweets. We can now restrict who can leave a reply under our tweet but the feature is limited.

If you're running a big account that provides content, you don't want to block all conversations under your tweet because after all, not all people are trolls. If you only want to reduce the number of trolls, in this case people who post fancam (related or not), this script is the answer. Another case of use of this script is if you want to read the conversations under a tweet WITHOUT having to go through all those fancam posts.

This is what this script does: blocking all people who post fancam (video) under a certain tweet. You'll need a Twitter developer account (apply here) with 'Read and Write permission'.

Paste the link to the tweet then run the script. Full script can be found on my github.

Blocking twitter trolls (1): Blocking retweeters

Azka — Mon, 24 Aug 2020 12:29:38 +0000

Twitter is one of the most popular platforms for fandom activities, especially my fandom, and for the past years I've found a lot of mutuals I like to interact with. On the other hand, the deeper you're into the fandom, the more you encounter unreasonable tweets. You have many options: ignore, mute, or block. If you wish not to engage with any of those people in the future, you can automatically block (or mute) the original poster and the people who retweeted with the script below.

You'll need a Twitter developer account (apply here) with 'Read and Write' permission. The full script is posted on my github.

Note that by blocking them all at once you've shut down all the opportunities for any type of interaction with those people. I personally would like to read people's sentiment about a topic, but you'd know a troll tweet when you spot one. What this script does is block those people, but the judgment is yours.

DEV Community: Azka

Setting Up Github Pages using Jupyter Book

Jupyter Book vs Jekyll for Github Pages

Live Version

Jupyter Book Tutorials

Assumptions

Steps

Notes

Setting Up Github Pages site with Jekyll Tutorial

Live Version

Assumptions

Tutorial Links

Steps

Install Ruby and Jekyll

Prepare Your Project Folder

Make Changes to Your Pages

Push Existing Project to Github

Use a Customized CSS

Deploy Flask Application to PythonAnywhere or Heroku (without database)

PythonAnywhere (paling mudah)

Heroku

Making a simple twitter autobase bot that posts incoming DM

Description

Features

1. Posts incoming DM that contains the trigger word.

2. Posts incoming DM with media. Three ways of sending a post containing media through DM:

3. Cuts post longer than 280 characters and posts them as a thread. The result is something like this.

4. Sends a DM back for each incoming DM.

6. Makes a log for each run into test.log file.

What do you need

How to run this

How do I automate the script

Notes

Questions

Blocking twitter trolls (2): Blocking fancams

Blocking twitter trolls (1): Blocking retweeters

6. Makes a log for each run into `test.log` file.